Computer Science ›› 2024, Vol. 51 ›› Issue (11A): 231100160-6.doi: 10.11896/jsjkx.231100160

• Image Processing & Multimedia Technology • Previous Articles     Next Articles

Automatic Sleep Staging Based on Multimodal Data and Fusion Deep Network

ZHAO Ruonan, LI Duo, SONG Jiangling, ZHANG Rui   

  1. Medical Big Data Research Center,Northwest University,Xi'an 710127,China
  • Online:2024-11-16 Published:2024-11-13
  • About author:ZHAO Ruonan,born in 1999,postgra-duate.Her main research interests include medical big data analytics and deep learning.
    ZHANG Rui,born in 1971,Ph.D,professor,Ph.D supervisor.Her main research interests include medical big data analytics,machine learning and neural computational modeling.
  • Supported by:
    National Natural Science Foundation of China(12071369,62006189),Natural Science Foundation of Shannxi Province,China(2021JQ-430,2023-JC-QN-0028) and China Postdoctoral Science Foundation(2022M722580).

Abstract: Accurate sleep staging is an important basis for evaluating sleep quality and diagnosing related diseases.Aiming at the differences between electroencephalogram(EEG) andElectrooculogram in different stages of sleep,this paper proposes a new feature fusion deep network based on EEG and EOG,called MAFSNet,to realize automatic sleep staging.Specifically,we first design two different one-dimensional convolutional neural networks to extract effective sleep features from EEG and EOG signals.Se-condly,an adaptive feature fusion module is constructed to assign different weights to the features according to their contribution degree.By enhancing discriminant features and suppressing irrelevant features,an adaptive fusion feature containing multi-modal sleep information is obtained.Then,the time series related information in the sleep stage transition rule is learned using the bidirectional long short-term memory network.Finally,the public data set Sleep-EDF is used to verify the effectiveness of the proposed model to achieve five-stage sleep staging.The results show that the proposed method has high classification performance in sleep staging,with the accuracy of 94.1%,Cohen's Kappa coefficient of 88.2% and Macro-averaged F1-score of 81.9%,in which the recall rate of N1 and REM sleep stages is significantly increased to 64.6% and 93.5%,respectively.

Key words: Automatic sleep staging, EEG signal, EOG signal, Deep neural network, Adaptive feature fusion

CLC Number: 

  • TP391
[1]PHAN H,MIKKELSEN K.Automatic sleep staging of EEGsignals:recent development,challenges,and future directions[J].Physiological Measurement,2022,43(4):04TR01.
[2]EFE E,OZSEN S.CoSleepNet:Automated sleep staging using a hybrid CNN-LSTM network on imbalanced EEG-EOG datasets[J].Biomedical Signal Processing and Control,2023,80:104299.
[3]China Sleep Research Society.2023 Chinese nationalhealthysleep white paper[R].Beijing:China Sleep Research Society,2023.
[4]BERRY R B,BROOKS R,GAMALDO C E,et al.The AASMmanual for the scoring of sleep and associated events[J].Rules,Terminology and Technical Specifications,Darien,Illinois,American Academy of Sleep Medicine,2012,176:2012.
[5]PEI W,LI Y,SIULY S,et al.A hybrid deep learning scheme for multi-channel sleep stage classification[J].Computers,Materials and Continua,2022,71(1):889-905.
[6]XU X,CHEN C,MENG K,et al.NAMRTNet:Automatic classification of sleep stages based on imp-roved ResNet-TCN network and attention mechanism[J].Applied Sciences,2023,13(11):6788.
[7]PHAN H,MIKKELSEN K,CHÉN O Y,et al.Sleeptransfor-mer:Automatic sleep staging with interpretability and uncertainty quantification[J].IEEE Transactions on Biomedical Engineering,2022,69(8):2456-2467.
[8]LI Y,PENG C,ZHANG Y,et al.Adversarial Learning forSemisupervised Pediatric Sleep Staging with Single-EEG Channel[J].Methods,2022,204:84-91.
[9]ZHU T,LUO W,YU F.Multi-branch convolutional neural network for automatic sleep stage classification with embedded stage refinement and residual attention channel fusion[J].Sensors,2020,20(22):6592.
[10]ABDOLLAHPOUR M,REZAII T Y,FARZAMNIA A,et al.Transfer learning convolutional neural network for sleep stage classification using two-stage data fusion framework[J].IEEE Access,2020,8:180618-180632.
[11]JIA Z,CAI X,ZHENG G,et al.SleepPrintNet:A multivariate multimodal neural network based on physiological time-series for automatic sleep staging[J].IEEE Transactions on Artificial Intelligence,2020,1(3):248-257.
[12]ZHAO C,LI J,GUO Y,et al.SleepContextNet:A temporal context network for automatic sleep staging based single-channel EEG[J].Computer Methods and Programs in Biomedicine,2022,220:106806.
[13]HUANG G,MA F.TrustSleepNet:A trustable deep multimodal network for sleep stage classifycation[C]//2022 IEEE-EMBS International Conference on Biomedical and Health Informatics(BHI).IEEE,2022:1-4.
[14]SAJAD M,FATEMEH A,RAJENDRA U A.SleepEEGNet:Automated sleep stage scoring with sequence to sequence deep learning approach[J].PloS One,2019,14(5):e0216456.
[15]LIN T Y,GOYAL P,GIRSHICK R,et al.Focal loss for dense object detection[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:2980-2988.
[16]GOLDBERGER A L,AMARAL L A N,GLASS L,et al.Phy-sioBank,physiotoolkit,and physionet:components of a new research resource for complex physiologic signals[J].Circulation,2000,101(23):e215-e220.
[17]KEMP B,ZWINDERMAN A H,TUK B,et al.Analysis of a sleep-dependent neuronal feedback loop:the slow-wave microcontinuity of the EEG[J].IEEE Transactions on Biomedical Engineering,2000,47(9):1185-1194.
[18]DUAN L,LI M,WANG C,et al.A novel sleep staging network based on data adaptation and multimodal fusion[J].Frontiers in Human Neuroscience,2021,15:727139.
[19]YILDIRIM O,BALOGLU U B,ACHARYA U R.A DeepLearning Model for Automated Sleep Stages Classification Using PSG Signals[J].International Journal of Environmental Research and Public Health,2019,16(4):599.
[20]FENG L X,LI X,WANG H Y,et al.Automatic sleep staging algorithm based on time attention mechanism[J].Frontiers in Human Neuroscience,2021,15:692054.
[21]JOE M J,PYO S C.Classification of Sleep Stage with Biosignal Images Using Convolutional Neural Networks[J].Applied Sciences,2022,12(6):3028.
[22]LI M,CHEN H,LIU Y,et al.4s-SleepGCN:Four-Stream GraphConvolutional Networks for Sleep Stage Classification[J].IEEE Access,2023,6(17):70612-70634.
[1] ZHU Fukun, TENG Zhen, SHAO Wenze, GE Qi, SUN Yubao. Semantic-guided Neural Network Critical Data Routing Path [J]. Computer Science, 2024, 51(9): 155-161.
[2] HAN Bing, DENG Lixiang, ZHENG Yi, REN Shuang. Survey of 3D Point Clouds Upsampling Methods [J]. Computer Science, 2024, 51(7): 167-196.
[3] XU Xiaohua, ZHOU Zhangbing, HU Zhongxu, LIN Shixun, YU Zhenjie. Lightweight Deep Neural Network Models for Edge Intelligence:A Survey [J]. Computer Science, 2024, 51(7): 257-271.
[4] ZHU Jin, TAO Chuanqi, GUO Hongjing. Test Input Prioritization Approach Based on DNN Model Output Differences [J]. Computer Science, 2024, 51(6A): 230600121-8.
[5] LI Wenting, XIAO Rong, YANG Xiao. Improving Transferability of Adversarial Samples Through Laplacian Smoothing Gradient [J]. Computer Science, 2024, 51(6A): 230800025-6.
[6] ZHANG Mingze, LI Yi, WU Wenyuan, SHI Mingquan, WANG Zhengjiang. FCTNet:Bus Arrival Time Prediction Method Based on Dual Domain Deep Learning [J]. Computer Science, 2024, 51(11A): 231000180-7.
[7] LUO Ruiqi, YAN Jinlin, HU Xinrong, DING Lei. EEG Emotion Recognition Based on Multiple Directed Weighted Graph and ConvolutionalNeural Network [J]. Computer Science, 2023, 50(6A): 220600128-8.
[8] BAI Zhixu, WANG Hengjun, GUO Kexiang. Adversarial Examples Generation Method Based on Image Color Random Transformation [J]. Computer Science, 2023, 50(4): 88-95.
[9] YIN Haitao, WANG Tianyou. Image Denoising Algorithm Based on Deep Multi-scale Convolution Sparse Coding [J]. Computer Science, 2023, 50(4): 133-140.
[10] RAO Dan, SHI Hongwei. Study on Air Traffic Flow Recognition and Anomaly Detection Based on Deep Clustering [J]. Computer Science, 2023, 50(3): 121-128.
[11] WANG Xiangwei, HAN Rui, Chi Harold LIU. Hierarchical Memory Pool Based Edge Semi-supervised Continual Learning Method [J]. Computer Science, 2023, 50(2): 23-31.
[12] REN Shuyao, SONG Jiangling, ZHANG Rui. Early Screening Method for Depression Based on EEG Signal [J]. Computer Science, 2023, 50(11A): 221100139-6.
[13] DONG Hongbin, HAN Shuang, FU Qiang. Geo-sensory Time Series Prediction Based on Joint Model of Auto Regression and Deep NeuralNetwork [J]. Computer Science, 2023, 50(11): 41-48.
[14] WEI Hui, CHEN Ze-mao, ZHANG Li-qiang. Anomaly Detection Framework of System Call Trace Based on Sequence and Frequency Patterns [J]. Computer Science, 2022, 49(6): 350-355.
[15] JIAO Xiang, WEI Xiang-lin, XUE Yu, WANG Chao, DUAN Qiang. Automatic Modulation Recognition Based on Deep Learning [J]. Computer Science, 2022, 49(5): 266-278.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!