Computer Science ›› 2024, Vol. 51 ›› Issue (2): 182-188.doi: 10.11896/jsjkx.230400184

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Cross-scene Gesture Recognition Based on Point Cloud Trajectories and Compressed Doppler

ZHANG Hongwang, ZHOU Rui, CHENG Yu, LIU Chenxu   

  1. University of Electronic Science and Technology of China,Chengdu 610054,China
  • Received:2023-04-27 Revised:2023-11-16 Online:2024-02-15 Published:2024-02-22
  • About author:ZHANG Hongwang,born in 1998,postgraduate.His main research interests include wireless sensing and deep lear-ning.ZHOU Rui,born in 1974,Ph.D,asso-ciate professor,is a member of CCF(No.52289M).Her main research intere-sts include pervasive computing,Internet of things,and artificial intelligence.
  • Supported by:
    Sichuan Province Science and Technology Service(2021GFW027), Sichuan Province Science and Technology Plan(2022YFSY0006), Sichuan Province Science and Technology Plan(2023YFSY0007) and Science and Technology Plan Project of Luzhou City, Sichuan Province(2022-XDY-192).

Abstract: Millimeter wave radar can be used for various sensing tasks,such as activity recognition,gesture recognition,heart rate perception.Among them,gesture recognition is a research hotspot,which can realize contactless human-computer interaction.Most existing studies on gesture recognition make use of point cloud or range-Doppler for pattern recognition through neural networks to achieve sensing.However,there are some problems.Firstly,the robustness of these methods is poor.The changes of the user and his/her location affect the received millimeter wave signals,causing the accuracy of the sensing model to reduce.Secondly,these methods input the complete range-Doppler map into the neural network,which makes the model complicated and makes it difficult for the model to focus on the sensing task,because there are many unrelated regions to the sensing task.To solve these problems,this paper first builds the gesture trajectory from multiple continuous frames of point cloud,and then cuts and compresses the multiple continuous range-Doppler maps to obtain the two-dimensional local Doppler map.Finally,the features are extracted from the point cloud trajectory and the two-dimensional local Doppler map respectively by the neural networks,concatenated and classified by a fully-connected neural network.Experiments show that the proposed method focuses on gestures and can achieve a recognition accuracy of 98%,and can achieve a recognition accuracy of 93% for new users and 92% for new locations in the cases of user changes and location changes,better than the state of the art.

Key words: Millimeter wave radar, Gesture recognition, Across scenarios, Location independent, User independent

CLC Number: 

  • TP391
[1]SHU Y,WANG Y,YANG X.et al.An improved denoisingmethod for eye blink detection using automotive millimeter wave radar[J/OL].EURASIP J.Adv.Signal Process.2022,9.https://asp-eurasipjournals.springeropen.com/articles/10.1186/s13634-022-00841-y.
[2]ZHAO P,LU C,WANG B,et al.Heart Rate Sensing with a Robot Mounted mmWave Radar[C]//IEEE International Confe-rence on Robotics and Automation.2020:2812-2818.
[3]LING Z,ZHOU W,REN Y.Non-Contact Heart Rate Monitoring Based on Millimeter Wave Radar[J].IEEE Access,2022,10:74033-74044.
[4]ZHAO P,LU X,WANG J,et al.Human tracking and identification through a millimeter wave radar[J].Ad Hoc Networks,2021,116(5):102475.1-102475.11.
[5]WANG S,SONG J,LIEN J,et al,Interacting with soli:Exploring fine-grained dynamic gesture recognition in the radio-frequency spectrum[C]//Proceedings of 29th Annual ACM Symposium on User Interface Software and Technology.2016:851-860.
[6]LIU H,DONG Z,SUN Y,et al.M-Gesture:Person-Independent Real-Time In-Air Gesture Recognition Using Commodity Millimeter Wave Radar[J].IEEE Internet of Things Journal,2022,9(5):3397-3415.
[7]LIU H,WANG Y,ZHOU A,et al.Real-time Arm Gesture Re-cognition in Smart Home Scenarios via Millimeter Wave Sensing[C]//Proceedings of ACM on Interactive,Mobile,Wearable and Ubiquitous Technology.2020.
[8]WEI H,LI Z,GALVAN A,et al.IndexPen:Two-Finger TextInput with Millimeter-Wave Radar[C]//Proceedings of ACM Interactive,Mobile,Wearable and Ubiquitous Technology.2022.
[9]SINGH L,YOU S,JEONG B J,et al.Remote Estimation ofBlood Pressure Using Millimeter-Wave Frequency-Modulated Continuous-Wave Radar[J].Sensors,2023,23(14):6517.
[10]MAOW,HE J,QIU L.CAT:High-precision acoustic motion tracking[C]//Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking.2016:69-81.
[11]WATANABE H,TERADA T.Improving ultrasound-based gesture recognition using a partially shielded single microphone[C]//Proceedings of the 2018 ACM International Symposium on Wearable Computers.2018:9-16.
[12]GAO R,ZHANG M,ZHANG J,et al.Towards Position-Independent Sensing for Gesture Recognition with Wi-Fi[J].Proceedings of the ACM on Interactive,Mobile,Wearable and Ubiquitous Technologies,2021,5(2):1-28.
[13]XIAO R,LIU J,HAN J,et al.OneFi:One-Shot Recognition for Unseen Gesture via COTS WiFi[C]//Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems.2021:206-219.
[14]DOAN H G,VU H,TRAN T H.Recognition of hand gestures from cyclic hand movements using spatial-temporal features[C]//Proceedings of the 6th International Symposium on Information and Communication Technology.2015:260-267.
[15]PIGOU L,VAN DEN OORD A,DIELEMAN S,et al.Beyond temporal pooling:Recurrence and temporal convolutions for gesture recognition in video[C]//International Journal of Computer Vision.2018:430-439.
[16]ZHAO W,WANG T,TAN A.Nighttime Pedestrian Detection Based on a Fusion of Visual Information and Millimeter-Wave Radar[J].IEEE Access,2023,11:68439-68451.
[17]XIA Z,DING G,WANG H.Person Identification With Millimeter-Wave Radar in Realistic Smart Home Scenarios[J].IEEE Geoscience and Remote Sensing Letters,2022,19:1-5.
[18]GAO Z,ALI L,WANG C.Real-Time Non-Contact MillimeterWave Radar-Based Vital Sign Detection[J].Sensors,2022,22(19):7560.
[19]ZHANG Y,PENG L.Dynamic Gesture Recognition ModelBased on Millimeter-Wave Radar With ResNet-18 and LSTM [J].Frontiers in Neurorobotics,2022,16:903197.
[1] WANG Ruiping, WU Shihong, ZHANG Meihang, WANG Xiaoping. Review of Vision-based Neural Network 3D Dynamic Gesture Recognition Methods [J]. Computer Science, 2024, 51(4): 193-208.
[2] WANG Tianran, WANG Qi, WANG Qingshan. Transfer Learning Based Cross-object Sign Language Gesture Recognition Method [J]. Computer Science, 2023, 50(6A): 220300232-5.
[3] WANG Chi, CHANG Jun. CSI Cross-domain Gesture Recognition Method Based on 3D Convolutional Neural Network [J]. Computer Science, 2021, 48(8): 322-327.
[4] LIU Liang, PU Hao-yang. Real-time LSTM-based Multi-dimensional Features Gesture Recognition [J]. Computer Science, 2021, 48(8): 328-333.
[5] LIU Xiao, YUAN Guan, ZHANG Yan-mei, YAN Qiu-yan, WANG Zhi-xiao. Hand Gesture Recognition Based on Self-adaptive Multi-classifiers Fusion [J]. Computer Science, 2020, 47(7): 103-110.
[6] JING Yu, QI Rui-hua, LIU Jian-xin, LIU Zhao-xia. Gesture Recognition Algorithm Based on Improved Multiscale Deep Convolutional Neural Network [J]. Computer Science, 2020, 47(6): 180-183.
[7] HAN Xiao, ZHANG Jing, LI Yue-long. Gesture Recognition Based on Hand Geometric Distribution Feature [J]. Computer Science, 2019, 46(6A): 246-249.
[8] LI Yu, CHAI Guo-zhong, LU Chun-fu, TANG Zhi-chuan. On-line sEMG Hand Gesture Recognition Based on Incremental Adaptive Learning [J]. Computer Science, 2019, 46(4): 274-279.
[9] LIU Jia-hui, WANG Yu-jie, LEI Yi. CSI Gesture Recognition Method Based on LSTM [J]. Computer Science, 2019, 46(11A): 283-288.
[10] SONG Yi-fan, ZHANG Peng, LIU Li-bo. Human-machine Interaction System with Vision-based Gesture Recognition [J]. Computer Science, 2019, 46(11A): 570-574.
[11] CHEN Tian-tian, YAO Huang, ZUO Ming-zhang, TIAN Yuan, YANG Meng-ting. Review of Dynamic Gesture Recognition Based on Depth Information [J]. Computer Science, 2018, 45(12): 42-51.
[12] LIU Zhe and LI Zhi. Research and Development of Computer-aided Requirements Engineering Tool Based on Multi-modal Interaction Technologies [J]. Computer Science, 2017, 44(4): 177-181.
[13] WANG Hai-peng, GONG Yan, LIU Wu, LI Ze and ZHANG Si-mei. Study on Spatial-Temporal Multiscale Adaptive Method of Gesture Recognition [J]. Computer Science, 2017, 44(12): 287-291.
[14] YI Jing-guo, CHENG Jiang-hua and KU Xi-shu. Review of Gestures Recognition Based on Vision [J]. Computer Science, 2016, 43(Z6): 103-108.
[15] YU Ze-sheng, CUI Wen-hua and SHI Tian-wei. Application and Research on Gesture Recognition by Kinect Sensors [J]. Computer Science, 2016, 43(Z11): 568-571.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!