计算机科学 ›› 2026, Vol. 53 ›› Issue (3): 78-87.doi: 10.11896/jsjkx.250500025

• 基于AGI技术的智能信息系统 • 上一篇    下一篇

基于双分支融合与分段域适应迁移学习的疲劳驾驶检测

李泽群1, 丁飞1,2,3   

  1. 1 南京邮电大学物联网学院 南京 210003
    2 南京邮电大学现代邮政学院 南京 210003
    3 南京邮电大学智慧交通学院 南京 210003
  • 收稿日期:2025-05-09 修回日期:2025-09-10 发布日期:2026-03-12
  • 通讯作者: 丁飞(dingfei@njupt.edu.cn)
  • 作者简介:(18896709562@163.com)
  • 基金资助:
    国家自然科学基金(62471241);江苏省高等学校基础科学(自然科学)研究重大项目(25KJA510003);江苏省“六大人才高峰”高层次人才资助项目(DZXX-008)

Fatigue Driving Detection Based on Dual-branch Fusion and Segmented Domain AdaptationTransfer Learning

LI Zequn1, DING Fei1,2,3   

  1. 1 School of Internet of Things, Nanjing University of Posts and Telecommunications, Nanjing 210003, China
    2 School of Modern Posts, Nanjing University of Posts and Telecommunications, Nanjing 210003, China
    3 School of Intelligent Transportation, Nanjing University of Posts and Telecommunications, Nanjing 210003, China
  • Received:2025-05-09 Revised:2025-09-10 Online:2026-03-12
  • About author:LI Zequn,born in 1997,postgraduate.His main research interest is intelligent assisted driving systems.
    DING Fei,born in 1981,Ph.D,professor,is a senior member of CCF(No.H2242S).His main research interests include crowd and participatory sen-sing,cyber-physical systems.
  • Supported by:
    National Natural Science Foundation of China(62471241),Major Project of the Natural Science Foundation of the Jiangsu Higher Education Institutions of China(25KJA510003) and “Six Talent Peaks” High Level Talent Funding Project of Jiangsu Province(DZXX-008).

摘要: 疲劳驾驶是引发交通事故的重要原因之一。针对实际场景中摄像头角度、环境光等因素导致的特征提取不足和不同数据下模型适应性差的问题,提出了一种新型基于迁移学习的疲劳驾驶检测框架。该框架通过设计卷积神经网络与Transformer双分支特征提取与融合结构,实现CNN与Transformer的优势互补,增强了模型的特征表征能力,充分提取了驾驶员的局部与全局面部特征。为提高模型在源域与目标域之间的自适应能力,框架采取分段域适应策略,在特征提取阶段采用对抗域适应和多核最大均值差异(MK-MMD)策略,并在特征融合阶段进一步引入MK-MMD和最小类别混淆损失(MCC),使模型充分适应不同数据。在两个具有显著特征差异的数据集上的实验结果表明,该框架在目标域上的检测准确率达到了93.3%(A为源域,B为目标域)和75.1%(B为源域,A为目标域),显著提升了模型的适应性与鲁棒性。

关键词: 深度学习, 域适应, 迁移学习, 疲劳检测, 卷积网络, Transformer

Abstract: Fatigue driving is one of the leading causes of traffic accidents.To address the issues of insufficient feature extraction caused by factors such as camera angles and environmental lighting,as well as poor model adaptability across different datasets,this paper proposes a novel transfer learning-based fatigue driving detection framework.The framework employs a dual-branch feature extraction and fusion architecture combining CNN and Transformer,leveraging their complementary strengths to enhance feature representation and comprehensively capture both local and global facial features of drivers.To improve the model’s adaptive capability between source and target domains,a segmented domain adapt-ation strategy is adopted.Adversarial domain adaptation and multi-kernel maximum mean discrepancy(MK-MMD) are applied during the feature extraction stage,while MK-MMD and minimum class confusion(MCC) loss are further introduced during the feature fusion stage to enhance cross-domain adapt-ability.Experimental results on two datasets with significant feature disparities demonstrate that the proposed framework achieves6 detection accuracies of 93.3%(A→B) and 75.1%(B→A) on the target domain,significantly improving the model’s adaptability and robustness.

Key words: Deep learning, Domain adaptation, Transfer learning, Fatigue detection, Convolutional network, Transformer

中图分类号: 

  • TP391.9
[1]AHMED S K,MOHAMMED M G,ABDULQADIR S O.Road traffic accidental injuries and deaths:A neglected global health issue[J].Health Science Reports,2023,6(5):1240.
[2]AN J,CAI Q,SUN X,et al.Attention-based cross-frequencygraph convolutional network for driver fatigue estimation[J].Cognitive Neurodynamics,2024,18(5):3181-3194.
[3]JIA H J,XIAO Z J,JI P.End-to-end fatigue driving EEG signal detection model based on improved temporal-graph convolution network[J].Computers in Biology and Medicine,2023,152:106431.
[4]PENG B,ZHANG Y,WANG M,et al.TA-MFFNet:Multi-feature fusion network for EEG analysis and driving fatigue detection based on time domain network and attention network[J].Computational Biology and Chemistry,2023,104:107863.
[5]CAI S X,DU C K,ZHOU S Y,et al.Detection of Fatigue Dri-ving State Based on Vehicle Operating Data[J].Transportation Systems Engineering and Information,2020,20(4):77.
[6]CHEN L W,CHEN H M.Driver Behavior Monitoring andWarning With Dangerous Driving DetectionBased on the Internet of Vehicles[J].IEEE Transactions on Intelligent Transportation Systems,2021,22(11):7232-7241.
[7]XU H Z,HAO D S,XU X T,et al.Expressway small object detection algorithm based on deep learning[J].Journal of Jilin University(Engineering and Technology Edition),2025,55(6):2003-2014.
[8]LI T Z,ZHANG T C,LI C,et al.Driver Fatigue Detection Based on Facial Inverted Pendulum Model and Information Entropy[J].Transportation Systems Engineering and Information,2023,23(5):24.
[9]CHEN L,WEI Z.Research on railway dispatcher fatigue detection method based on deep learning with multi-feature fusion[J].Electronics,2023,12(10):2303.
[10]LIU M Z,XU X,HU J,et al.Real time detection of driver fatigue based on CNN-LSTM[J].IET Image Processing,2022,16(2):576-595.
[11]YI Y,ZHOU Z,ZHANG W,et al.Fatigue detection algorithm based on eye multi feature fusion[J].IEEE Sensors Journal,2023,23(7):7949-7955.
[12]CELONA L,MAMMANA L,BIANCO S,et al.A multi-taskCNN framework for driver face monitoring[C]//2018 IEEE 8th International Conference on Consumer Electronics-Berlin(ICCE-Berlin).IEEE,2018:1-4.
[13]HUANG R,WANG Y,LI Z,et al.RF-DCM:Multi-granularity deep convolutional model based on feature recalibration and fusion for driver fatigue detection[J].IEEE Transactions on Intelligent Transportation Systems,2020,23(1):630-40.
[14]LYU J,YUAN Z,CHEN D.Long-term multi-granularity deep framework for driver drowsiness detection[J].arXiv:1801.02325,2018.
[15]MANDAL B,LI L,WANG G S,et al.Towards detection of bus driver fatigue based on robust visual analysis of eye state[J].IEEE Transactions on Intelligent Transportation Systems,2016,18(3):545-557.
[16]CATALBAS M C,CEGOVNIK T,SODNIK J,et al.Driver fatigue detection based on saccadic eye movements[C]//2017 10th International Conference on Electrical and Electronics Enginee-ring(ELECO).IEEE,2017:913-917.
[17]HARIRI B,ABTAHI S,SHIRMOHAMMADI S,et al.A yaw-ning measurement method to detect driver drowsiness[C]//IEEE International Instrumentation and Measurement Techno-logy Conference.2012.
[18]NAKAMURA T,MAEJIMA A,MORISHIMA S.Detection of Driver’s drowsy facial expression[C]//2013 2nd IAPR Asian Conference on Pattern Recognition.IEEE,2013:749-753.
[19]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[20]LI X,CAO L,ZHANG G,et al.Driver fatigue detection based on convolutional neural network and face alignment for edge computing device[C]//Proceedings of the Institution of Mechanical Engineers,Part D:Journal of Automobile Engineering.2021:2699-711.
[21]ZHAO G,HE Y,YANG H,et al.Research on fatigue detection based on visual features[J].IET Image Processing.2022,16(4):1044-53.
[22]ZHAO L,WANG Z,ZHANG G,et al.Driver drowsiness recognition via transferred deep 3D convolutional network and state probability vector[J].Multimedia Tools and Applications,2020,79(35):26683-26701.
[23]ALAMEEN S A,ALHOTHALI A M.A Lightweight DriverDrowsiness Detection System Using 3DCNN With LSTM[J].Computer Systems Science & Engineering,2023,44(1):895-912.
[24]DOSOVITSKIY A.An image is worth 16×16 words:Transfor-mers for image recognition at scale[J].arXiv:2010.11929,2020.
[25]PENG Z,HUANG W,GU S,et al.Conformer:Local featurescoupling global representations for visual recognition[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:367-376.
[26]WANG M,DENG W.Deep visual domain adaptation:A survey[J].Neurocomputing,2018,312:135-153.
[27]TENG S H,HUANG L L,ZHANG W.Dual Strategies andConfidence-based Domain Adaptation Learning[J].Journal of Chinese Computer Systems.2025,46(5):1135-1146.
[28]LIU Z,LIN Y,CAO Y,et al.Swin transformer:Hierarchical vision transformer using shifted windows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:10012-10022.
[29]LONG M,CAO Y,WANG J,et al.Learning transferable fea-tures with deep adaptation networks[C]//International Confe-rence on Machine Learning.PMLR,2015:97-105.
[30]GANIN Y,USTINOVA E,AJAKAN H,et al.Domain-adversarial training of neural networks[J].Journal of Machine Lear-ning Research,2016,17(59):1-35.
[31]Driver-Drowsiness-Detection-Gk0ws-Jii6r Dataset[EB/OL].https://gitee.com/lzq320147/driver-drowsiness-detection-gk0ws-jii6r.
[32]Drowsy-Driving-Det2[EB/OL].https://gitee.com/lzq320147/drowsy-driving-det2.
[33]LONG M,CAO Z,WANG J,et al.Conditional adversarial do-main adaptation[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems.2018:1647-1657.
[34]PEI Z,CAO Z,LONG M,et al.Multi-adversarial domain adaptation[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2018.
[35]PRABHU V,KHARE S,KARTIK D,et al.Sentry:Selectiveentropy optimization via committee consistency for unsupervised domain adaptation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:8558-8567.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!