Computer Science ›› 2024, Vol. 51 ›› Issue (12): 181-189.doi: 10.11896/jsjkx.231200170

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Millimeter Wave Radar Human Activity Recognition Algorithm Based on Feature Fusion

HAN Chong, FAN Weibei, GUO Ao   

  1. College of Computer, Nanjing University of Posts and Telecommunications, Nanjing 210023, China
  • Received:2023-12-25 Revised:2024-05-06 Online:2024-12-15 Published:2024-12-10
  • About author:HAN Chong,born in 1985,Ph.D,asso-ciate professor,master supervisor,is a member of CCF(No.C3132M).His main research interests include wireless sensing and RF computing.
  • Supported by:
    National Natural Science Foundation of China(62272242).

Abstract: The human activity recognition method based on millimeter-wave radar captures the electromagnetic wave signals of human activities in non-contact way for recognition.It is not easily interfered by smoke and light,which has a certain degree of privacy protection,and has become a research hotspot at present.However,the existing algorithms have some problems,such as single feature input,complex model structure,and insufficient generalization ability verification.A human activity recognition algorithm with two stream feature fusion convolutional neural network is proposed,named 2S-FCNN,which uses the residual neural network equipped with attention mechanism as the backbone network,inputs the time-distance image and the time-velocity image in parallel,and uses the feature weighted score fusion method to fuse the features for classification and recognition,so as to achieve a high recognition accuracy.A set of in-depth comparative experiments are conducted with other existing algorithms on both public and self built datasets,and the experimental results show that the proposed algorithm has good performance in recognition rate and generalization ability.

Key words: Millimeter wave radar, Human activity recognition, Feature fusion, Attention mechanism

CLC Number: 

  • TP391.41
[1]DING C Y,LIU K,LI G,et al.Spatio-Temporal Weighted Posture Motion Features for Human Skeleton Action Recognition Research[J].Chinese Journal of Computers,2020,43(1):29-40.
[2]ZHANG X P,JI J H,WANG L,et al.Overview of video based human abnormal behavior recognition and detection methods[J].Control and Decision,2022,37(1):14-27.
[3]DENG M L,GAO Z D,LI L,et al.Overview of Human Behavior Recognition based on Deep Learning[J].Computer Engineering and Applications,2022,58(13):14-26.
[4]MAQSOOD R,BAJWA U I,SALEEM G,et al.Anomaly recognition from surveillance videos using 3D convolution neural network[J].Multimedia Tools and Applications,2021,80(12):18693-18716.
[5]LIANG J,ZHU H,ZHANG E,et al.Stargazer:A transformer-based driver action detection system for intelligent transportation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:3160-3167.
[6]SIMONYAN K,ZISSERMAN A.Two-stream convolutionalnetworks for action recognition in videos[J].arXiv:1406.2199,2014.
[7]TRAN D,BOURDEV L,FERGUS R,et al.Learning spatiotemporal features with 3d convolutional networks[C]//Proceedings of the IEEE International Conference on Computer Vision.2015:4489-4497.
[8]YIN J,YANG Q,PAN J J.Sensor-based abnormal human-acti-vity detection[J].IEEE Transactions on Knowledge and Data Engineering,2008,20(8):1082-1090.
[9]MA Y,ARSHAD S,MUNIRAJU S,et al.Location-and person-independent activity recognition with WiFi,deep neural networks,and reinforcement learning[J].ACM Transactions on Internet of Things,2021,2(1):1-25.
[10]YAO L,SHENG Q Z,LI X,et al.Compressive representation for device-free activity recognition with passive RFID signal strength[J].IEEE Transactions on Mobile Computing,2017,17(2):293-306.
[11]TAYLOR W,DASHTIPOUR K,SHAH S A,et al.Radar sen-sing for activity classification in elderly people exploiting micro-doppler signatures using machine learning[J].Sensors,2021,21(11):3881.
[12]SAEED U,SHAH S Y,SHAH S A,et al.Discrete human acti-vity recognition and fall detection by combining FMCW RADAR data of heterogeneous environments for independent assistive living[J].Electronics,2021,10(18):2237.
[13]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[14]HAN T,KANG W,CHOI G.IR-UWB sensor based fall detection method using CNN algorithm[J].Sensors,2020,20(20):5948.
[15]VICTORIA A H,MARAGATHAM G.Activity recognition of FMCW radar human signatures using tower convolutional neural networks[J/OL].Wireless Networks,2021:1-17.https://doi.org/10.1007/s11276-021-02670-7.
[16]ABDU F J,ZHANG Y,DENG Z.Activity classification based on feature fusion of FMCW radar human motion micro-Doppler signatures[J].IEEE Sensors Journal,2022,22(9):8648-8662.
[17]SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].arXiv:1409.1556,2014.
[18]KRIZHEVSKY A,SUTSKEVER I,HINTON G E.Imagenetclassification with deep convolutional neural networks[J].Communications of the ACM,2017,60(6):84-90.
[19]ALI A S,RADWAN A G,SOLIMAN A M.Fractional OrderButterworth Filter:Active and Passive Realizations[J].IEEE Journal on Emerging and Selected Topics in Circuits and Systems,2013,3(3):346-354.
[20]WOO S,PARK J,LEE J Y,et al.Cbam:Convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision(ECCV).2018:3-19.
[1] LI Xin, PU Yuanyuan, ZHAO Zhengpeng, LI Yupan, XU Dan. Image Arbitrary Style Transfer via Artistic Aesthetic Enhancement [J]. Computer Science, 2024, 51(9): 129-139.
[2] LI Yunchen, ZHANG Rui, WANG Jiabao, LI Yang, WANG Ziqi, CHEN Yao. Re-parameterization Enhanced Dual-modal Realtime Object Detection Model [J]. Computer Science, 2024, 51(9): 162-172.
[3] HU Pengfei, WANG Youguo, ZHAI Qiqing, YAN Jun, BAI Quan. Night Vehicle Detection Algorithm Based on YOLOv5s and Bistable Stochastic Resonance [J]. Computer Science, 2024, 51(9): 173-181.
[4] LIU Qian, BAI Zhihao, CHENG Chunling, GUI Yaocheng. Image-Text Sentiment Classification Model Based on Multi-scale Cross-modal Feature Fusion [J]. Computer Science, 2024, 51(9): 258-264.
[5] LI Zhe, LIU Yiyang, WANG Ke, YANG Jie, LI Yafei, XU Mingliang. Real-time Prediction Model of Carrier Aircraft Landing Trajectory Based on Stagewise Autoencoders and Attention Mechanism [J]. Computer Science, 2024, 51(9): 273-282.
[6] LIU Qilong, LI Bicheng, HUANG Zhiyong. CCSD:Topic-oriented Sarcasm Detection [J]. Computer Science, 2024, 51(9): 310-318.
[7] YAO Yao, YANG Jibin, ZHANG Xiongwei, LI Yihao, SONG Gongkunkun. CLU-Net Speech Enhancement Network for Radio Communication [J]. Computer Science, 2024, 51(9): 338-345.
[8] LIU Sichun, WANG Xiaoping, PEI Xilong, LUO Hangyu. Scene Segmentation Model Based on Dual Learning [J]. Computer Science, 2024, 51(8): 133-142.
[9] ZHANG Rui, WANG Ziqi, LI Yang, WANG Jiabao, CHEN Yao. Task-aware Few-shot SAR Image Classification Method Based on Multi-scale Attention Mechanism [J]. Computer Science, 2024, 51(8): 160-167.
[10] WANG Qian, HE Lang, WANG Zhanqing, HUANG Kun. Road Extraction Algorithm for Remote Sensing Images Based on Improved DeepLabv3+ [J]. Computer Science, 2024, 51(8): 168-175.
[11] XIAO Xiao, BAI Zhengyao, LI Zekai, LIU Xuheng, DU Jiajin. Parallel Multi-scale with Attention Mechanism for Point Cloud Upsampling [J]. Computer Science, 2024, 51(8): 183-191.
[12] PU Bin, LIANG Zhengyou, SUN Yu. Monocular 3D Object Detection Based on Height-Depth Constraint and Edge Fusion [J]. Computer Science, 2024, 51(8): 192-199.
[13] ZHANG Junsan, CHENG Ming, SHEN Xiuxuan, LIU Yuxue, WANG Leiquan. Diversified Label Matrix Based Medical Image Report Generation [J]. Computer Science, 2024, 51(8): 200-208.
[14] WANG Chao, TANG Chao, WANG Wenjian, ZHANG Jing. Infrared Human Action Recognition Method Based on Multimodal Attention Network [J]. Computer Science, 2024, 51(8): 232-241.
[15] ZHANG Lu, DUAN Youxiang, LIU Juan, LU Yuxi. Chinese Geological Entity Relation Extraction Based on RoBERTa and Weighted Graph Convolutional Networks [J]. Computer Science, 2024, 51(8): 297-303.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!