计算机科学 ›› 2024, Vol. 51 ›› Issue (12): 181-189.doi: 10.11896/jsjkx.231200170

• 计算机图形学&多媒体 • 上一篇    下一篇

基于特征融合的毫米波雷达行为识别算法

韩崇, 樊卫北, 郭澳   

  1. 南京邮电大学计算机学院 南京 210023
  • 收稿日期:2023-12-25 修回日期:2024-05-06 出版日期:2024-12-15 发布日期:2024-12-10
  • 通讯作者: 韩崇(hc@njupt.edu.cn)
  • 基金资助:
    国家自然科学基金(62272242)

Millimeter Wave Radar Human Activity Recognition Algorithm Based on Feature Fusion

HAN Chong, FAN Weibei, GUO Ao   

  1. College of Computer, Nanjing University of Posts and Telecommunications, Nanjing 210023, China
  • Received:2023-12-25 Revised:2024-05-06 Online:2024-12-15 Published:2024-12-10
  • About author:HAN Chong,born in 1985,Ph.D,asso-ciate professor,master supervisor,is a member of CCF(No.C3132M).His main research interests include wireless sensing and RF computing.
  • Supported by:
    National Natural Science Foundation of China(62272242).

摘要: 基于毫米波雷达的人体行为识别方法以远程非接触的方式捕获人类活动的电磁波信号并进行识别,不受烟雾和光线等的干扰,具有一定的隐私保护性,是当前的一个研究热点。针对现有的算法存在特征输入单一、模型结构复杂、泛化能力验证性不够等问题,提出了基于双分支特征融合卷积神经网络(Two Steam Features Fusion Convolutional Neural Network,2S-FCNN),使用搭载注意力机制的残差神经网络作为骨干网络,并行输入时间距离图和时间速度图,采用特征加权分数融合的方式融合特征后进行分类识别,实现了较高的识别准确率。在公开数据集和自建数据集上与现有的其他算法进行了深入的对比实验,实验结果表明所提算法在识别率和泛化能力方面都具有良好的性能。

关键词: 毫米波雷达, 行为识别, 特征融合, 注意力机制

Abstract: The human activity recognition method based on millimeter-wave radar captures the electromagnetic wave signals of human activities in non-contact way for recognition.It is not easily interfered by smoke and light,which has a certain degree of privacy protection,and has become a research hotspot at present.However,the existing algorithms have some problems,such as single feature input,complex model structure,and insufficient generalization ability verification.A human activity recognition algorithm with two stream feature fusion convolutional neural network is proposed,named 2S-FCNN,which uses the residual neural network equipped with attention mechanism as the backbone network,inputs the time-distance image and the time-velocity image in parallel,and uses the feature weighted score fusion method to fuse the features for classification and recognition,so as to achieve a high recognition accuracy.A set of in-depth comparative experiments are conducted with other existing algorithms on both public and self built datasets,and the experimental results show that the proposed algorithm has good performance in recognition rate and generalization ability.

Key words: Millimeter wave radar, Human activity recognition, Feature fusion, Attention mechanism

中图分类号: 

  • TP391.41
[1]DING C Y,LIU K,LI G,et al.Spatio-Temporal Weighted Posture Motion Features for Human Skeleton Action Recognition Research[J].Chinese Journal of Computers,2020,43(1):29-40.
[2]ZHANG X P,JI J H,WANG L,et al.Overview of video based human abnormal behavior recognition and detection methods[J].Control and Decision,2022,37(1):14-27.
[3]DENG M L,GAO Z D,LI L,et al.Overview of Human Behavior Recognition based on Deep Learning[J].Computer Engineering and Applications,2022,58(13):14-26.
[4]MAQSOOD R,BAJWA U I,SALEEM G,et al.Anomaly recognition from surveillance videos using 3D convolution neural network[J].Multimedia Tools and Applications,2021,80(12):18693-18716.
[5]LIANG J,ZHU H,ZHANG E,et al.Stargazer:A transformer-based driver action detection system for intelligent transportation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:3160-3167.
[6]SIMONYAN K,ZISSERMAN A.Two-stream convolutionalnetworks for action recognition in videos[J].arXiv:1406.2199,2014.
[7]TRAN D,BOURDEV L,FERGUS R,et al.Learning spatiotemporal features with 3d convolutional networks[C]//Proceedings of the IEEE International Conference on Computer Vision.2015:4489-4497.
[8]YIN J,YANG Q,PAN J J.Sensor-based abnormal human-acti-vity detection[J].IEEE Transactions on Knowledge and Data Engineering,2008,20(8):1082-1090.
[9]MA Y,ARSHAD S,MUNIRAJU S,et al.Location-and person-independent activity recognition with WiFi,deep neural networks,and reinforcement learning[J].ACM Transactions on Internet of Things,2021,2(1):1-25.
[10]YAO L,SHENG Q Z,LI X,et al.Compressive representation for device-free activity recognition with passive RFID signal strength[J].IEEE Transactions on Mobile Computing,2017,17(2):293-306.
[11]TAYLOR W,DASHTIPOUR K,SHAH S A,et al.Radar sen-sing for activity classification in elderly people exploiting micro-doppler signatures using machine learning[J].Sensors,2021,21(11):3881.
[12]SAEED U,SHAH S Y,SHAH S A,et al.Discrete human acti-vity recognition and fall detection by combining FMCW RADAR data of heterogeneous environments for independent assistive living[J].Electronics,2021,10(18):2237.
[13]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[14]HAN T,KANG W,CHOI G.IR-UWB sensor based fall detection method using CNN algorithm[J].Sensors,2020,20(20):5948.
[15]VICTORIA A H,MARAGATHAM G.Activity recognition of FMCW radar human signatures using tower convolutional neural networks[J/OL].Wireless Networks,2021:1-17.https://doi.org/10.1007/s11276-021-02670-7.
[16]ABDU F J,ZHANG Y,DENG Z.Activity classification based on feature fusion of FMCW radar human motion micro-Doppler signatures[J].IEEE Sensors Journal,2022,22(9):8648-8662.
[17]SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].arXiv:1409.1556,2014.
[18]KRIZHEVSKY A,SUTSKEVER I,HINTON G E.Imagenetclassification with deep convolutional neural networks[J].Communications of the ACM,2017,60(6):84-90.
[19]ALI A S,RADWAN A G,SOLIMAN A M.Fractional OrderButterworth Filter:Active and Passive Realizations[J].IEEE Journal on Emerging and Selected Topics in Circuits and Systems,2013,3(3):346-354.
[20]WOO S,PARK J,LEE J Y,et al.Cbam:Convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision(ECCV).2018:3-19.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!