计算机科学

计算机科学 ›› 2025, Vol. 52 ›› Issue (6A): 240400169-9.doi: 10.11896/jsjkx.240400169

• 图像处理&多媒体技术 • 上一篇    下一篇

基于Transformer和PointNet++的毫米波雷达人体姿态估计

李阳1, 刘毅2, 李浩3,4, 张刚2, 徐明枫1, 郝崇清2   

  1. 1 中国信息通信研究院 北京 100191
    2 河北科技大学电气工程学院 石家庄 050018
    3 中科院雄安创新研究院 河北 雄安新区 070001
    4 河北省认知智能重点实验室 河北 雄安新区 070001
  • 出版日期:2025-06-16 发布日期:2025-06-12
  • 通讯作者: 郝崇清(haochongqing@hebust.edu.cn)
  • 作者简介:(liyang3@caict.ac.cn)
  • 基金资助:
    国家重点研发计划(2022YFB2804402)

Human Pose Estimation Using Millimeter Wave Radar Based on Transformer and PointNet++

LI Yang1, LIU Yi2, LI Hao3,4, ZHANG Gang2, XU Mingfeng1, HAO Chongqing2   

  1. 1 China Academy of Information and Communications Technology,Beijing 100191,China
    2 School of Electrical Engineering,Hebei University of Science and Technology,Shijiazhuang 050018,China
    3 Chinese Academy of Sciences,Xiongan Institute of Innovation,Xiongan,Hebei 070001,China
    4 Hebei Provincial Key Laboratory of Cognitive Intelligence,Xiongan,Hebei 070001,China
  • Online:2025-06-16 Published:2025-06-12
  • About author:LI Yang,born in 1993,postgraduate,intermediate engineer.His main research interests include wireless AI for 6G,integrated sensing and communications.
    HAO Chongqing,born in 1981,Ph.D,associate professor.His main research interests include machine vision and biomimetic robots.
  • Supported by:
    National Key R&D Program of China(2022YFB2804402).

PDF (PC)

59

摘要： 人体姿态估计作为动作识别领域中的研究热题被广泛地应用在医疗、安防和监控等方面,对推动相关行业的智能化发展具有重要意义。但目前基于图像的人体姿态估计对环境要求较高且隐私性差。基于此,提出了一种基于毫米波雷达点云的人体姿态估计方法,该方法使用PointNet++对毫米波雷达点云进行特征提取,与基于CNN的姿态估计方法相比,其在各关节点的MSE,MAE,RMSE值更低。此外,为了解决毫米波雷达点云稀疏的问题,使用了一种多帧点云拼接策略,以增加点云的数量,其中以拼接三帧点云为输入的模型相比于原始模型的MSE和MAE值分别降低了0.22 cm和0.72 cm,有效地缓解了点云过于稀疏的问题。最后,为了充分利用不同点云之间的时序特征,将Transformer与PointNet++相结合,并通过消融实验证明了多帧点云拼接策略和加入Transformer结构这两种方法的有效性,其MSE和MAE两个指标值分别达到了0.59 cm和5.41 cm,为实现性能更优的射频人体姿态估计提供了一种新思路。

关键词: 人体姿态估计, 毫米波雷达, PointNet++, 点云数据, Transformer

Abstract: Human pose estimation,as a hot research topic in the field of action recognition,is widely applied in medical,security,and monitoring fields,and is of great significance for promoting the intelligent development of related industries.However,currently image-based human pose estimation has high environmental requirements and poor privacy.Based on this,a human pose estimation method based on millimeter wave radar point cloud is proposed.This method uses PointNet++ to extract features from millimeter wave radar point cloud.Compared with CNN based pose estimation methods,it has lower MSE,MAE,and RMSE values at each joint point.In addition,to solve the problem of sparse point clouds in millimeter wave radar,a multi frame point cloud stitching strategy is used to increase the number of point clouds.The model that concatenates three frame point clouds as input reduces the MSE and MAE values by 0.22 cm and 0.72 cm respectively compared to the original model,effectively alleviating the problem of excessively sparse point clouds.Finally,in order to fully utilize the temporal features between different point clouds,Transformer is combined with PointNet++,and the effectiveness of the multi frame point cloud stitching strategy and the addition of Transformer structure are demonstrated through ablation experiments.The MSE and MAE values reaches 0.59 cm and 5.41 cm respectively,providing a new approach for achieving better performance RF human pose estimation.

Key words: Human pose estimation, Millimeter wave radar, PointNet++, Point cloud data, Transformer

中图分类号: 

  • TP391
[1]AN S,OGRAS U Y.Mars:Mmwave-Based Assistive Rehabilitation System for Smart Healthcare[J].ACM Transactions on Embedded Computing Systems(TECS),2021,20(5s):1-22.
[2]ZHENG J,SHI X,GORBAN A,et al.Multi-Modal 3D Human Pose Estimation With 2D Weak Supervision in Autonomous Driving[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:4478-4487.
[3]ÇETINKAYA M,ACARMAN T.Driver Activity RecognitionUsing Deep Learningand Human Pose Estimation[C]//2021 International Conference on INnovations in Intelligent SysTems and Applications(INISTA).IEEE,2021:1-5.
[4]MURPHY-CHUTORIAN E,TRIVEDI M M.Head Pose Estimationand Augmented Reality Tracking:An Integrated System and Evaluation for Monitoring Driver Awareness[J].IEEE Transactions on Intelligent Transportation Systems,2010,11(2):300-311.
[5]CHEN Y,WANG Z,PENG Y,et al.Cascaded Pyramid Net-workfor Multi-Person Pose Estimation[C]//Proceedings of The IEEE Conference on Computer Vision and Pattern Recognition.2018:7103-7112.
[6]SUN K,XIAO B,LIU D,et al.Deep High-Resolution Representation Learning for Human Pose Estimation[C]//Proceedings of The IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:5693-5703.
[7]INSAFUTDINOV E,PISHCHULIN L,ANDRES B,et al.DeeperCut:A Deeper,Stronger,and Faster Multi-Person Pose Estimation Model[C]//Proceedings of the European Conference on Computer Vision.2016:34-50.
[8]CAO Z,SIMON T,WEI S E,et al.Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields[C]//Proceedings of The IEEE Conference on Computer Vision and Pattern Recognition.2017:7291-7299.
[9]LI Y P,LIU T T,ZHANG L.A human action recognition method based on deep learning[J].Chinese Journal of Computer Application Research,2020,37(1):304-307,316.
[10]ZHAO M,LI T,ABU A M,et al.Through-Wall Human Pose Estimation Using Radio Signals[C]//Proceedings of The IEEE Conference on Computer Vision and Pattern Recognition.2018:7356-7365.
[11]FAN L,LI T,YUAN Y,et al.In-Home Daily-Life Captioning Using Radio Signals[C]//Computer Vision-ECCV 2020:16th European Conference,Glasgow,UK,August 23-28,2020,Proceedings,Part II 16.Springer International Publishing,2020:105-123.
[12]LI G,ZHANG Z,YANG H,et al.Capturing Human Pose Using Mmwave Radar[C]//2020 IEEE International Conference on Pervasive Computing and Communications Workshops(PerCom Workshops).IEEE,2020:1-6.
[13]CAO Z,DING W,CHEN R,et al.A Joint Global-Local Network for Human Pose Estimation with Millimeter Wave Radar[J].IEEE Internet of Things Journal,2022,10(1):434-446.
[14]JIN F,SENGUPTA A,CAO S,et al.Mmwave Radar PointCloud Segmentation Using Gmm in Multimodal Traffic Monitoring[C]//2020 IEEE International Radar Conference(RADAR).IEEE,2020:732-737.
[15]AN S,OGRAS U Y.Fast And Scalable Human Pose Estimation Using Mmwave Point Cloud[C]//Proceedings of The 59th ACM/IEEE Design Automation Conference.2022:889-894.
[16]QI C R,YI L,SU H,et al.Pointnet++:Deep Hierarchical Feature Learning On Point Sets In A Metric Space[J].Advances in Neural Information Processing Systems,2017,30:5099-5108.
[17]VASWANI A,SHAZEER N,PARMAR N,et al.Attention IsAll You Need[J].Advances in Neural Information Processing Systems,2017,30:5998-6008.
[18]LIN J H,YIN H B,XIONG R P.Indoor Personnel Positioning Method based on FMCW Radar Sensors[J].Chinese Journal of Sensors and Microsystems,2021,40(11):50-53.
[19]ZHONG J X,JIN L N.A robust adaptive clustering method for millimeter wave radar target point clouds[J].Chinese Journal of Science,Technology and Engineering,2022,22(5):1936-1943.
[20]YIN H B,XU Z M.Human Motion Trajectory Detection System based on FMCW Radar[J].Chinese Journal of Sensors and Microsystems,2020,39(9):116-118.
[21]QI C R,SU H,MO K,et al.Pointnet:Deep Learningon PointSets for 3D Classification and Segmentation[C]//Proceedings of The IEEE Conference on Computer Vision and Pattern Recognition.2017:652-660.
[22]INSTRUMENTS T.IWR1443BOOST[OL].[2024-04-11].https://www.ti.com/tool/IWR1443BOOST.
[23]INSTRUMENTS T.DCA1000EVM[OL].[2024-04-11].ht-tps://www.ti.com.
[24]Asus.Primesense carmine 1.09T[OL].[2024-04-11].http://xtionprolive.com/primesense-carmine-1.09.
[25]WANG R Z.Continuous human pose construction based on millimeter wave radar point cloud[D].Shanghai:Donghua University,2024.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发