Computer Science ›› 2024, Vol. 51 ›› Issue (6): 247-255.doi: 10.11896/jsjkx.230300033

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Early-stage Fatigue Detection Based on Frequency Domain Information of Eye Features

HUO Xingxing1, HU Ruimin2, LI Yixin1   

  1. 1 School of Cyber Engineering,Xidian University,Xi’an 710126,China
    2 Hangzhou Institution of Technology,Xidian University,Hangzhou 310000,China
  • Received:2023-03-06 Revised:2023-07-30 Online:2024-06-15 Published:2024-06-05
  • About author:HUO Xingxing,born in 1998,postgra-duate.Her main research interests include pattern recognition and machine learning.
    HU Ruimin,born in 1964,Ph.D,professor,Ph.D supervisor.His main research interests include behavior pattern mi-ning and complex relationship analysis.
  • Supported by:
    National Natural Science Foundation of China(U22A2035) and Nanning Scientific Research and Technological Development Plans(20231042).

Abstract: The fatigue of baggage X-ray security inspector is an important cause of false and missed inspection.Previous work in this field mostly focused on detecting extreme fatigue with explicit signs such as yawning,nodding off and prolonged eye closure.However,for security inspectors,such explicit signs may not appear until only before an accident,and it is too late to detect fatigue.Thus,there is significant value in detecting fatigue at an early stage,to warn the occurrence of fatigue in time.Due to the subtle facial performance characteristics of early-stage fatigue,the irreversibility of time-domain parameters leads to its inability for complete representations.To solve this problem,an early-stage fatigue detection method for baggage X-ray security inspectors based on the frequency domain information of eye features is proposed,which converts the original time domain information into a more expressive frequency domain feature space.It firstly obtained the eye aspect ratio series through the facial detection algorithm,then the time-domain features are transformed into frequency-domain space for analysis to mine more subtle features.Finally,HM-LSTM is used for training and verification. Experiment is conducted on the dataset UTA-RLDD.The results show that the proposed architecture improves the recognition rate of early-stage fatigue by 2%,demonstrating that frequency domain features have better expression ability than time domain features.

Key words: Eye aspect ratio, Frequency domain feature, Time-Frequency transform, Early-stage fatigue, Airport security

CLC Number: 

  • TP391
[1]AKERSTEDT T,GILLBERG M.Subjective and objectives leepiness in the active individual[J].International Journal of Neuroscience,1990,52(1/2):29-37.
[2]SMITH A P,ALLEN P H,WADSWORTH E J K.Seafarer fatigue:The Cardiff researchprogramme[J/OL].https://orca.cardiff.ac.uk/id/eprint/52615/.
[3]JIA W,PENG H,RUAN N,et al.WiFind:Driver fatigue detection with fine-grained Wi-Fi signal features[J].IEEE Transactions on Big Data,2018,6(2):269-282.
[4]LIU P,CHI H L,LI X,et al.Effects of dataset characteristics on the performance of fatigue detection for crane operators using hybrid deep neural networks[J].Automation in Construction,2021,132:103901.
[5]LI G,CHUNG W Y.A context-aware EEG headset system for early detection of driver drowsiness[J].Sensors,2015,15(8):20873-20893.
[6]GHODDOOSIAN R,GALIB M,ATHITSOS V.A realisticdataset and baseline temporal model for early drowsiness detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.2019.
[7]VAN SEGBROECK M,TSIARTAS A,NARAYANAN S S.A robust frontend for VAD:exploiting contextual,discriminative and spectral cues of human voice[C]//INTERSPEECH.2013:704-708.
[8]ZHANG R,LI P H,LIANG K W,et al.Voice Activity Detection by Jo1int MRCG and MFCC Features with Robustness Detection based GRU Networks[C]//2021 IEEE International Conference on Consumer Electronics-Taiwan(ICCE-TW).IEEE,2021.
[9]ABDULLAH S,ZAMANI M,DEMOSTHENOUS A.A dis-crete wavelet transform-based voice activity detection and noise classification with sub-band selection[C]//2021 IEEE International Symposium on Circuits and Systems(ISCAS).IEEE,2021:1-5.
[10]LIU F,DEMOSTHENOUS A.A computation efficient voice activity detector for low signal-to-noise ratio in hearing aids[C]//2021 IEEE International Midwest Symposium on Circuits and Systems(MWSCAS).IEEE,2021:524-528.
[11]KAIDA K,TAKAHASHI M,AKERSTEDT T,et al.Validation of theKarolinska sleepiness scale against performance and EEG variables[J].Clinical Neurophysiology,2006,117(7):1574-1581.
[12]XU C,WANG X,CHEN X.Modeling fatigue level by driver’s lane-keeping indicators[C]//ICTE 2013:Safety,Speediness,Intelligence,Low-Carbon,Innovation.2013:2282-2288.
[13]ZHANG R,ZHU T J,ZOU Z L,et al.A Survey of Driver Fatigue Detection Methods[J].Computer Engineering and Application,2022,58(21):53-66.
[14]ZHU X,ZHENG W L,LU B L,et al.EOG-based drowsiness detection using convolutional neural networks[C]//2014 International Joint Conference on Neural Networks(IJCNN).IEEE,2014:128-134.
[15]JIAO Y,DENG Y,LUO Y,et al.Driver sleepiness detectionfrom EEG and EOG signals using GAN and LSTM networks[J].Neurocomputing,2020,408:100-111.
[16]NIU J,WANG F,SONG J Q,et al.Video Emotion Recognition of Face Features and Pulse Signal Features[J].Journal of Chongqing University of Technology(Natural Science),2021,35(8):144-150.
[17]CHEN S,SUN Y,ZHANG H,et al.Speech Fatigue DetectionBased on Deep Learning[C]//Journal of Physics:Conference Series.IOP Publishing,2022.
[18]XIE Y,LI F,WU Y,et al.Real-time detection for drowsy dri-ving via acoustic sensing on smartphones[J].IEEE Transactions on Mobile Computing,2020,20(8):2671-2685.
[19]TAO H,ZHANG G,ZHAO Y,et al.Real-time driver fatigue detection based on face alignment[C]//Ninth International Conference on Digital Image Processing(ICDIP 2017).SPIE,2017:6-11.
[20]BEKHOUCHE S E,RUICHEK Y,DORNAIKA F.Driverdrowsiness detection in video sequences using hybrid selection of deep features[J].Knowledge-Based Systems,2022,252:109436.
[21]YADAV S,RAI A.Frequency and temporal convolutional attention for text-independent speaker recognition[C]//IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP 2020).IEEE,2020:6794-6798.
[22]CECH J,SOUKUPOVA T.Real-time eye blink detection using facial landmarks[J/OL].https://dspace.cvut.cz/bitstream/handle/10467/64839/F3-DP-2016-Soukupova-Tereza-SOUKUPOVA_DP_2016.pdf;jsessionid=3C81CDFAD46C8E54026CF6FB17C9BCE9?sequence=-1.
[23]LEE Y,MIN J,HAN D K,et al.Spectro-temporal attention-based voice activity detection[J].IEEE Signal Processing Letters,2019,27:131-135.
[24]DING K,ZHENG S,QI X,et al.Acoustic Target Recognition Based on MFCC and SVM[C]//Man-Machine-Environment System Engineering:Proceedings of the 22nd International Conference on MMESE.Singapore:Springer Nature Singapore,2022:418-423.
[25]TADESSE E,SHENG W,LIU M.Driver drowsiness detection through HMM based dynamic modeling[C]//2014 IEEE International Conference on Robotics and Automation(ICRA).IEEE,2014:4003-4008.
[26]REDDY B,KIM Y H,YUN S,et al.Real-time driver drowsiness detection for embedded system using model compression of deepneural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops.2017:121-128.
[27]PARK S,PAN F,KANG S,et al.Driver drowsiness detectionsystem based on feature representation learning using various deep networks[C]//Computer Vision-ACCV 2016 Workshops:ACCV 2016 International Workshops,Taipei,Taiwan,Revised Selected Papers,Part III.Springer,2017:154-164.
[28]WENG C H,LAI Y H,LAI S H.Driver drowsiness detection via a hierarchical temporal deep belief network[C]//Computer Vision-ACCV 2016 Workshops:ACCV 2016 International Workshops,Taipei,Taiwan,Revised Selected Papers,Part III 13.Springer,2017:117-133.
[29]ABTAHI S,OMIDYEGANEH M,SHIRMOHAMMADI S,et al.YawDD:A yawning detection dataset[C]//Proceedings of the 5th ACM Multimedia Systems Conference.2014:24-28.
[1] MA Xin, JI Lixin, LI Shaomei. Forgery Face Detection Based on Multi-scale Transformer Fusing Multi-domain Information [J]. Computer Science, 2023, 50(10): 112-118.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!