计算机科学 ›› 2024, Vol. 51 ›› Issue (2): 182-188.doi: 10.11896/jsjkx.230400184

• 计算机图形学&多媒体 • 上一篇    下一篇

基于点云轨迹和压缩多普勒的跨场景手势识别

张宏旺, 周瑞, 程宇, 刘辰旭   

  1. 电子科技大学信息与软件工程学院 成都610054
  • 收稿日期:2023-04-27 修回日期:2023-11-16 出版日期:2024-02-15 发布日期:2024-02-22
  • 通讯作者: 周瑞(ruizhou@uestc.edu.cn)
  • 作者简介:(zhw6770@qq.com)
  • 基金资助:
    四川省科技服务业(2021GFW027);四川省科技计划(2022YFSY0006);四川省科技计划(2023YFSY0007);泸州市科技计划项目(2022-XDY-192)

Cross-scene Gesture Recognition Based on Point Cloud Trajectories and Compressed Doppler

ZHANG Hongwang, ZHOU Rui, CHENG Yu, LIU Chenxu   

  1. University of Electronic Science and Technology of China,Chengdu 610054,China
  • Received:2023-04-27 Revised:2023-11-16 Online:2024-02-15 Published:2024-02-22
  • About author:ZHANG Hongwang,born in 1998,postgraduate.His main research interests include wireless sensing and deep lear-ning.ZHOU Rui,born in 1974,Ph.D,asso-ciate professor,is a member of CCF(No.52289M).Her main research intere-sts include pervasive computing,Internet of things,and artificial intelligence.
  • Supported by:
    Sichuan Province Science and Technology Service(2021GFW027), Sichuan Province Science and Technology Plan(2022YFSY0006), Sichuan Province Science and Technology Plan(2023YFSY0007) and Science and Technology Plan Project of Luzhou City, Sichuan Province(2022-XDY-192).

摘要: 毫米波雷达能够用于各种感知任务,如活动识别、手势识别、心率感知等。手势识别作为其中的研究热点,可实现无接触人机交互。目前大多数手势识别研究使用点云或距离多普勒图通过神经网络进行识别感知,但是这些方法存在一些问题。首先,这些方法鲁棒性较差,被感知人员或其位置发生变化都会影响接收到的毫米波信号,降低感知精度。其次,这些方法将完整的距离多普勒图输入神经网络进行识别,由于图中存在较多与感知任务无关的区域,模型复杂且难以专注于感知任务。为解决这些问题,首先从连续多帧点云数据中建立手势轨迹,然后将连续多帧距离多普勒图进行局部切割并压缩获得二维局部多普勒图,最后将点云轨迹和二维局部多普勒图分别经过神经网络特征提取后,对特征进行拼接,通过全连接神经网络进行分类。实验结果表明,所提方法专注于手势,能够达到98%的识别准确率,在人员变化和位置变化情况下对新用户和在新位置的识别准确率分别能够达到93%和92%,高于现有方法。

关键词: 毫米波雷达, 手势识别, 跨场景, 位置无关, 人无关

Abstract: Millimeter wave radar can be used for various sensing tasks,such as activity recognition,gesture recognition,heart rate perception.Among them,gesture recognition is a research hotspot,which can realize contactless human-computer interaction.Most existing studies on gesture recognition make use of point cloud or range-Doppler for pattern recognition through neural networks to achieve sensing.However,there are some problems.Firstly,the robustness of these methods is poor.The changes of the user and his/her location affect the received millimeter wave signals,causing the accuracy of the sensing model to reduce.Secondly,these methods input the complete range-Doppler map into the neural network,which makes the model complicated and makes it difficult for the model to focus on the sensing task,because there are many unrelated regions to the sensing task.To solve these problems,this paper first builds the gesture trajectory from multiple continuous frames of point cloud,and then cuts and compresses the multiple continuous range-Doppler maps to obtain the two-dimensional local Doppler map.Finally,the features are extracted from the point cloud trajectory and the two-dimensional local Doppler map respectively by the neural networks,concatenated and classified by a fully-connected neural network.Experiments show that the proposed method focuses on gestures and can achieve a recognition accuracy of 98%,and can achieve a recognition accuracy of 93% for new users and 92% for new locations in the cases of user changes and location changes,better than the state of the art.

Key words: Millimeter wave radar, Gesture recognition, Across scenarios, Location independent, User independent

中图分类号: 

  • TP391
[1]SHU Y,WANG Y,YANG X.et al.An improved denoisingmethod for eye blink detection using automotive millimeter wave radar[J/OL].EURASIP J.Adv.Signal Process.2022,9.https://asp-eurasipjournals.springeropen.com/articles/10.1186/s13634-022-00841-y.
[2]ZHAO P,LU C,WANG B,et al.Heart Rate Sensing with a Robot Mounted mmWave Radar[C]//IEEE International Confe-rence on Robotics and Automation.2020:2812-2818.
[3]LING Z,ZHOU W,REN Y.Non-Contact Heart Rate Monitoring Based on Millimeter Wave Radar[J].IEEE Access,2022,10:74033-74044.
[4]ZHAO P,LU X,WANG J,et al.Human tracking and identification through a millimeter wave radar[J].Ad Hoc Networks,2021,116(5):102475.1-102475.11.
[5]WANG S,SONG J,LIEN J,et al,Interacting with soli:Exploring fine-grained dynamic gesture recognition in the radio-frequency spectrum[C]//Proceedings of 29th Annual ACM Symposium on User Interface Software and Technology.2016:851-860.
[6]LIU H,DONG Z,SUN Y,et al.M-Gesture:Person-Independent Real-Time In-Air Gesture Recognition Using Commodity Millimeter Wave Radar[J].IEEE Internet of Things Journal,2022,9(5):3397-3415.
[7]LIU H,WANG Y,ZHOU A,et al.Real-time Arm Gesture Re-cognition in Smart Home Scenarios via Millimeter Wave Sensing[C]//Proceedings of ACM on Interactive,Mobile,Wearable and Ubiquitous Technology.2020.
[8]WEI H,LI Z,GALVAN A,et al.IndexPen:Two-Finger TextInput with Millimeter-Wave Radar[C]//Proceedings of ACM Interactive,Mobile,Wearable and Ubiquitous Technology.2022.
[9]SINGH L,YOU S,JEONG B J,et al.Remote Estimation ofBlood Pressure Using Millimeter-Wave Frequency-Modulated Continuous-Wave Radar[J].Sensors,2023,23(14):6517.
[10]MAOW,HE J,QIU L.CAT:High-precision acoustic motion tracking[C]//Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking.2016:69-81.
[11]WATANABE H,TERADA T.Improving ultrasound-based gesture recognition using a partially shielded single microphone[C]//Proceedings of the 2018 ACM International Symposium on Wearable Computers.2018:9-16.
[12]GAO R,ZHANG M,ZHANG J,et al.Towards Position-Independent Sensing for Gesture Recognition with Wi-Fi[J].Proceedings of the ACM on Interactive,Mobile,Wearable and Ubiquitous Technologies,2021,5(2):1-28.
[13]XIAO R,LIU J,HAN J,et al.OneFi:One-Shot Recognition for Unseen Gesture via COTS WiFi[C]//Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems.2021:206-219.
[14]DOAN H G,VU H,TRAN T H.Recognition of hand gestures from cyclic hand movements using spatial-temporal features[C]//Proceedings of the 6th International Symposium on Information and Communication Technology.2015:260-267.
[15]PIGOU L,VAN DEN OORD A,DIELEMAN S,et al.Beyond temporal pooling:Recurrence and temporal convolutions for gesture recognition in video[C]//International Journal of Computer Vision.2018:430-439.
[16]ZHAO W,WANG T,TAN A.Nighttime Pedestrian Detection Based on a Fusion of Visual Information and Millimeter-Wave Radar[J].IEEE Access,2023,11:68439-68451.
[17]XIA Z,DING G,WANG H.Person Identification With Millimeter-Wave Radar in Realistic Smart Home Scenarios[J].IEEE Geoscience and Remote Sensing Letters,2022,19:1-5.
[18]GAO Z,ALI L,WANG C.Real-Time Non-Contact MillimeterWave Radar-Based Vital Sign Detection[J].Sensors,2022,22(19):7560.
[19]ZHANG Y,PENG L.Dynamic Gesture Recognition ModelBased on Millimeter-Wave Radar With ResNet-18 and LSTM [J].Frontiers in Neurorobotics,2022,16:903197.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!