计算机科学 ›› 2024, Vol. 51 ›› Issue (11A): 231100160-6.doi: 10.11896/jsjkx.231100160

• 图像处理&多媒体技术 • 上一篇    下一篇

基于多模态数据与融合深度网络的自动睡眠分期方法

赵若男, 李朵, 宋江玲, 张瑞   

  1. 西北大学医学大数据研究中心 西安 710127
  • 出版日期:2024-11-16 发布日期:2024-11-13
  • 通讯作者: 张瑞(rzhang@nwu.edu.cn)
  • 作者简介:(ruonan_zhao@yeah.net)
  • 基金资助:
    国家自然科学基金(12071369,62006189);陕西省自然科学基金(2021JQ-430,2023-JC-QN-0028);中国博士后科学基金(2022M722580)

Automatic Sleep Staging Based on Multimodal Data and Fusion Deep Network

ZHAO Ruonan, LI Duo, SONG Jiangling, ZHANG Rui   

  1. Medical Big Data Research Center,Northwest University,Xi'an 710127,China
  • Online:2024-11-16 Published:2024-11-13
  • About author:ZHAO Ruonan,born in 1999,postgra-duate.Her main research interests include medical big data analytics and deep learning.
    ZHANG Rui,born in 1971,Ph.D,professor,Ph.D supervisor.Her main research interests include medical big data analytics,machine learning and neural computational modeling.
  • Supported by:
    National Natural Science Foundation of China(12071369,62006189),Natural Science Foundation of Shannxi Province,China(2021JQ-430,2023-JC-QN-0028) and China Postdoctoral Science Foundation(2022M722580).

摘要: 准确的睡眠分期是进行睡眠质量评估及相关疾病诊断的重要依据。针对脑电信号(Electroencephalogram,EEG)和眼电信号(Electrooculogram,EOG) 在睡眠各阶段存在差异性,提出了一种用于实现自动睡眠分期的基于EEG和EOG的新型特征融合深度网络——MAFSNet。具体地,首先设计两种一维卷积神经网络分别用于提取EEG和EOG信号中的睡眠有效特征;其次,构建自适应的特征融合模块,根据特征的贡献程度赋予其不同的权值,通过增强判别特征和抑制无关特征,得到包含多模态睡眠信息的自适应融合特征;进而,采用双向长短期记忆网络学习睡眠阶段转换规则中的时间序列相关信息;最后,使用公开数据集Sleep-EDF验证所提模型实现五级睡眠分期的有效性。研究结果表明所提方法在睡眠分期中具有较高的分类性能,准确率、Kappa系数和MF1分数分别为94.1%,88.2%和81.9%,其中N1和REM睡眠阶段的召回率分别显著提升到64.6%和93.5%。

关键词: 自动睡眠分期, 脑电信号, 眼电信号, 深度神经网络, 自适应特征融合

Abstract: Accurate sleep staging is an important basis for evaluating sleep quality and diagnosing related diseases.Aiming at the differences between electroencephalogram(EEG) andElectrooculogram in different stages of sleep,this paper proposes a new feature fusion deep network based on EEG and EOG,called MAFSNet,to realize automatic sleep staging.Specifically,we first design two different one-dimensional convolutional neural networks to extract effective sleep features from EEG and EOG signals.Se-condly,an adaptive feature fusion module is constructed to assign different weights to the features according to their contribution degree.By enhancing discriminant features and suppressing irrelevant features,an adaptive fusion feature containing multi-modal sleep information is obtained.Then,the time series related information in the sleep stage transition rule is learned using the bidirectional long short-term memory network.Finally,the public data set Sleep-EDF is used to verify the effectiveness of the proposed model to achieve five-stage sleep staging.The results show that the proposed method has high classification performance in sleep staging,with the accuracy of 94.1%,Cohen's Kappa coefficient of 88.2% and Macro-averaged F1-score of 81.9%,in which the recall rate of N1 and REM sleep stages is significantly increased to 64.6% and 93.5%,respectively.

Key words: Automatic sleep staging, EEG signal, EOG signal, Deep neural network, Adaptive feature fusion

中图分类号: 

  • TP391
[1]PHAN H,MIKKELSEN K.Automatic sleep staging of EEGsignals:recent development,challenges,and future directions[J].Physiological Measurement,2022,43(4):04TR01.
[2]EFE E,OZSEN S.CoSleepNet:Automated sleep staging using a hybrid CNN-LSTM network on imbalanced EEG-EOG datasets[J].Biomedical Signal Processing and Control,2023,80:104299.
[3]China Sleep Research Society.2023 Chinese nationalhealthysleep white paper[R].Beijing:China Sleep Research Society,2023.
[4]BERRY R B,BROOKS R,GAMALDO C E,et al.The AASMmanual for the scoring of sleep and associated events[J].Rules,Terminology and Technical Specifications,Darien,Illinois,American Academy of Sleep Medicine,2012,176:2012.
[5]PEI W,LI Y,SIULY S,et al.A hybrid deep learning scheme for multi-channel sleep stage classification[J].Computers,Materials and Continua,2022,71(1):889-905.
[6]XU X,CHEN C,MENG K,et al.NAMRTNet:Automatic classification of sleep stages based on imp-roved ResNet-TCN network and attention mechanism[J].Applied Sciences,2023,13(11):6788.
[7]PHAN H,MIKKELSEN K,CHÉN O Y,et al.Sleeptransfor-mer:Automatic sleep staging with interpretability and uncertainty quantification[J].IEEE Transactions on Biomedical Engineering,2022,69(8):2456-2467.
[8]LI Y,PENG C,ZHANG Y,et al.Adversarial Learning forSemisupervised Pediatric Sleep Staging with Single-EEG Channel[J].Methods,2022,204:84-91.
[9]ZHU T,LUO W,YU F.Multi-branch convolutional neural network for automatic sleep stage classification with embedded stage refinement and residual attention channel fusion[J].Sensors,2020,20(22):6592.
[10]ABDOLLAHPOUR M,REZAII T Y,FARZAMNIA A,et al.Transfer learning convolutional neural network for sleep stage classification using two-stage data fusion framework[J].IEEE Access,2020,8:180618-180632.
[11]JIA Z,CAI X,ZHENG G,et al.SleepPrintNet:A multivariate multimodal neural network based on physiological time-series for automatic sleep staging[J].IEEE Transactions on Artificial Intelligence,2020,1(3):248-257.
[12]ZHAO C,LI J,GUO Y,et al.SleepContextNet:A temporal context network for automatic sleep staging based single-channel EEG[J].Computer Methods and Programs in Biomedicine,2022,220:106806.
[13]HUANG G,MA F.TrustSleepNet:A trustable deep multimodal network for sleep stage classifycation[C]//2022 IEEE-EMBS International Conference on Biomedical and Health Informatics(BHI).IEEE,2022:1-4.
[14]SAJAD M,FATEMEH A,RAJENDRA U A.SleepEEGNet:Automated sleep stage scoring with sequence to sequence deep learning approach[J].PloS One,2019,14(5):e0216456.
[15]LIN T Y,GOYAL P,GIRSHICK R,et al.Focal loss for dense object detection[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:2980-2988.
[16]GOLDBERGER A L,AMARAL L A N,GLASS L,et al.Phy-sioBank,physiotoolkit,and physionet:components of a new research resource for complex physiologic signals[J].Circulation,2000,101(23):e215-e220.
[17]KEMP B,ZWINDERMAN A H,TUK B,et al.Analysis of a sleep-dependent neuronal feedback loop:the slow-wave microcontinuity of the EEG[J].IEEE Transactions on Biomedical Engineering,2000,47(9):1185-1194.
[18]DUAN L,LI M,WANG C,et al.A novel sleep staging network based on data adaptation and multimodal fusion[J].Frontiers in Human Neuroscience,2021,15:727139.
[19]YILDIRIM O,BALOGLU U B,ACHARYA U R.A DeepLearning Model for Automated Sleep Stages Classification Using PSG Signals[J].International Journal of Environmental Research and Public Health,2019,16(4):599.
[20]FENG L X,LI X,WANG H Y,et al.Automatic sleep staging algorithm based on time attention mechanism[J].Frontiers in Human Neuroscience,2021,15:692054.
[21]JOE M J,PYO S C.Classification of Sleep Stage with Biosignal Images Using Convolutional Neural Networks[J].Applied Sciences,2022,12(6):3028.
[22]LI M,CHEN H,LIU Y,et al.4s-SleepGCN:Four-Stream GraphConvolutional Networks for Sleep Stage Classification[J].IEEE Access,2023,6(17):70612-70634.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!