计算机科学 ›› 2021, Vol. 48 ›› Issue (11): 226-233.doi: 10.11896/jsjkx.201200095

• 计算机图形学&多媒体 • 上一篇    下一篇

面向人机协同的物体姿态估计帧间稳定性优化方法

穆逢君1, 邱静1, 陈路锋2, 黄瑞2, 周林3, 于功敬3   

  1. 1 电子科技大学机械与电气工程学院 成都611731
    2 电子科技大学自动化工程学院 成都611731
    3 国防科技工业自动化测试创新中心 北京100041
  • 收稿日期:2020-12-11 修回日期:2021-03-14 出版日期:2021-11-15 发布日期:2021-11-10
  • 通讯作者: 邱静(qiujing@uestc.edu.cn)
  • 作者简介:mufengjun260@gmail.com
  • 基金资助:
    中央高校基本科研业务费专项资金(ZYGX2019Z010)

Optimization Method for Inter-frame Stability of Object Pose Estimation for Human-Machine Collaboration

MU Feng-jun1, QIU Jing1, CHEN Lu-feng2, HUANG Rui2, ZHOU Lin3, YU Gong-jing3   

  1. 1 School of Mechanical and Electrical Engineering,University of Electronic Science and Technology of China,Chengdu 611731,China
    2 School of Automation Engineering,University of Electronic Science and Technology of China,Chengdu 611731,China
    3 Innovation Center of Automated Testing for Science,Technology and Industry for National Defence,Beijing 100041,China
  • Received:2020-12-11 Revised:2021-03-14 Online:2021-11-15 Published:2021-11-10
  • About author:MU Feng-jun,born in 1997,postgra-duate.His main research interests include computer vision and human-machine collaboration.
    QIU Jing,born in 1977,Ph.D,associate professor.Her main research interests include exoskeleton robot and human factors engineering.
  • Supported by:
    Fundamental Research Funds for the Central Universities(ZYGX2019Z010).

摘要: 现有的物体姿态估计方法无法提供具有帧间稳定性的估计姿态,导致将其结果直接用于增强现实等可视化场景时会引起画面抖动,不适用于人机协同等应用场景。文中提出了一种包含多种方式的物体姿态估计优化方法,通过对原始姿态估计方法的损失函数的改进,并使用因果滤波的方法优化姿态估计结果,以获得具有稳定性的估计姿态。此外,为完善对姿态估计方法稳定程度的评价体系,文中提出了直接偏差距离DBD、方向反转率DRR与平均位移角ADA 3种评价指标,可以从多个角度对物体姿态估计方法的帧间稳定性进行评价。最后,使用YCB-STB数据集作为测试样本,并将所提方法与未经优化的原始方法进行对比测试。结果表明,所提方法可在不引入额外资源开销的情况下提高现有物体姿态估计方法的帧间稳定性,且对原始方法的准确率影响较小,满足了人机协同场景对物体姿态估计结果的需求。

关键词: 人机协同, 损失函数, 物体姿态估计, 因果滤波

Abstract: Existing object pose estimation methods cannot provide estimated poses with inter-frame stability.As a result,when the results are directly used in visualization scenarios such as augmented reality,it will cause screen jitter,so it's not suitable enough for application scenarios such as human-machine collaboration.This paper proposes an object pose estimation optimization method that includes multiple methods.By improving the loss function of the original pose estimation method and using causal filtering to optimize the pose estimation result,a stable estimated pose can be obtained.In addition,in order to consummate the eva-luation system of the degree of stability of the pose estimation method,this paper proposes three evaluation indicators:the direct deviation distance DBD,the direction reversal rate DRR and the average displacement angle ADA,which can evaluate the object pose estimation method from multiple viewpoints.Finally,the YCB-STB dataset is used to test,and the method is compared with the original method without optimization.The results show that the proposed method can improve the inter-frame stability of the existing object pose estimation methods without introducing additional resources,and has a small impact on the accuracy of the original method,which satisfies the requirement of object attitude estimation in human-machine collaborative scene.

Key words: Causal filtering, Human-machine collaboration, Loss function, Object pose estimate

中图分类号: 

  • TP242.6
[1]VAN KREVELEN D W F,POELMAN R.A survey of augmen-ted reality technologies,applications and limitations[J].International Journal of Virtual Reality,2010,9(2):1-20.
[2]LOUIS T,TROCCAZ J,ROCHET-CAPELLAN A,et al.Is it real? measuring the effect of resolution,latency,frame rate and jitter on the presence of virtual entities[C]//International Conference on Interactive Surfaces and Spaces(ISS).ACM,2019:5-16.
[3]DU G,WANG K,LIAN S,et al.Vision-based robotic grasping from object localization,object pose estimation to grasp estimation for parallel grippers:a review[J].Artificial Intelligence Review,2020,54(3):1677-1734.
[4]LEPETIT V,MORENO-NOGUER F,FUA P.EPnP:An Accurate O(n) Solution to the PnP Problem[J].International Journal of Computer Vision (IJCV),2009,81(2):155-166.
[5]ROSTEN E,DRUMMOND T.Fusing Points and Lines for High Performance Tracking[C]//International Conference on Computer Vision (ICCV).IEEE,2005:1508-1515.
[6]BAY H,TUYTELAARS T,VAN GOOL L.SURF:Speeded upRobust Features[C]//European Conference on Computer Vision(ECCV).Springer,2006:404-417.
[7]RUBLEE E,RABAUD V,KONOLIGE K,et al.ORB:An efficient alternative to SIFT or SURF[C]//International Confe-rence on Computer Vision(ICCV).IEEE,2011:2564-2571.
[8]DETONE D,MALISIEWICZ T,RABINOVICH A.SuperPoint:Self-Supervised Interest Point Detection and Description[C]//Conference on ComputerVision and Pattern Recognition Workshop(CVPRW).IEEE/CVF,2018:224-236.
[9]HU Y,HUGONOT J,FUA P,et al.Segmentation-Driven 6DObject Pose Estimation[C]//Conference on Computer Vision and Pattern Recognition(CVPR).IEEE/CVF,2019:3385-3394.
[10]YUAN Y,HOU J,NÜCHTER A,et al.Self-supervised Point Set Local Descriptors for Point Cloud Registration[J].arXiv:2003.05199,2020.
[11]HINTERSTOISSER S,HOLZER S,CAGNIART C,et al.Mul-timodal templates for real-time detection of texture-less objects in heavily cluttered scenes[C]//International Conference on Computer Vision(ICCV).IEEE,2011:858-865.
[12]XIANG Y,SCHMIDT T,NARAYANAN V,et al.PoseCNN:A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes[J].arXiv:1711.00199,2017.
[13]YANG J,LI H,CAMPBELL D,et al.Go-ICP:A globally optimal solution to 3D ICP point-set registration[J].IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI),2016,38(11):2241-2254.
[14]BESL P J,MCKAY N D.A Method for Registration of 3DShapes[C]//IEEE Transactions on Pattern Analysis and Machine Intelligence(PAMI).IEEE,1992:586-606.
[15]LU W,WAN G,ZHOU Y,et al.DeepICP:An end-to-end deep neural network for 3D point cloud registration[J].arXiv:1905.04153,2019.
[16]CHEN W,JIA X,CHANG H J,et al.G2L-Net:Global to Local Network for Real-time 6D Pose Estimation with Embedding Vector Features[C]//Conference on Computer Vision and Pattern Recognition(CVPR).IEEE/CVF,2020:4233-4242.
[17]TEJANI A,TANG D,KOUSKOURIDAS R,et al.Latent-Class Hough Forests for 3D Object Detection and Pose Estimation[C]//European Conference on Computer Vision(ECCV).Springer,2014:462-477.
[18]WANG C,XU D,ZHU Y,et al.DenseFusion:6D Object Pose Estimation by Iterative Dense Fusion[C]//Conference on Computer Vision and Pattern Recognition(CVPR).IEEE/CVF,2019:3343-3352.
[19]PENG S,LIU Y,HUANG Q,et al.PVNet:Pixel-Wise Voting Network for 6DoF Pose Estimation[C]//Conference on Computer Vision and Pattern Recognition(CVPR).IEEE/CVF,2019:4561-4570.
[20]HE Y,SUN W,HUANG H,et al.PVN3D:A Deep Point-Wise 3D Keypoints Voting Network for 6DoF Pose Estimation[C]//Conference on Computer Vision and Pattern Recognition(CVPR).IEEE/CVF,2020:11632-11641.
[21]WANG C,MARTÍN-MARTÍN R,XU D,et al.6-PACK:Category-level 6D Pose Tracker with Anchor-Based Keypoints[C]//International Conference on Robotics and Automation (ICRA).IEEE,2020:10059-10066.
[22]DENG X,XIANG Y,MOUSAVIAN A,et al.Self-supervised 6d object pose estimation for robot manipulation[C]//International Conference on Robotics and Automation (ICRA).IEEE,2020:3665-3671.
[23]CRIVELLARO A,RAD M,VERDIE Y,et al.Robust 3D Object Tracking from Monocular Images Using Stable Parts[J].IEEE Transactions on Pattern Analysis and Machine Intelligence(PAMI).2017,40(6):1465-1479.
[24]WEN B,MITASH C,REN B,et al.se(3)-TrackNet:Data-dri-ven 6D Pose Tracking by Calibrating Image Residuals in Synthe-tic Domains[J].arXiv:2007.13866,2020.
[25]DENG X,MOUSAVIAN A,XIANG Y,et al.PoseRBPF:ARao-Blackwellized Particle Filter for 6D Object Pose Tracking[J].arXiv:1905.09304,2019.
[26]GAO X,ZHANG T,YAN Q R,et al.14 Lectures on Visual SLAM:From Theory to Practice[M].Publishing House of Electronics Industry,2017.
[27]ZHAO H,SHI J,QI X,et al.Pyramid Scene Parsing Network[C]//Conference on Computer Vision and Pattern Recognition(CVPR).IEEE/CVF,2017:2881-2890.
[28]QI C R,SU H,MO K,et al.PointNet:Deep Learning on Point Sets for 3D Classification and Segmentation[C]//Conference on Computer Vision and Pattern Recognition(CVPR).IEEE/CVF,2017:652-660.
[29]KINGMA D P,BA J.ADAM:Adam:A Method for Stochastic Optimization[J].arXiv:1412.6980,2014.
[30]MARKLEY F L,CHENG Y,CRASSIDIS J L,et al.Averaging quaternions[J].Journal of Guidance,Control,and Dynamics,2007,30(4):1193-1197.
[31]CALLI B,SINGH A,WALSMAN A,et al.The YCB object and Model set:Towards common benchmarks for manipulation research[C]//International Conference on Advanced Robotics (ICAR).IEEE,2015:510-517.
[1] 孟月波, 穆思蓉, 刘光辉, 徐胜军, 韩九强.
基于向量注意力机制GoogLeNet-GMP的行人重识别方法
Person Re-identification Method Based on GoogLeNet-GMP Based on Vector Attention Mechanism
计算机科学, 2022, 49(7): 142-147. https://doi.org/10.11896/jsjkx.210600198
[2] 高荣华, 白强, 王荣, 吴华瑞, 孙想.
改进注意力机制的多叉树网络多作物早期病害识别方法
Multi-tree Network Multi-crop Early Disease Recognition Method Based on Improved Attention Mechanism
计算机科学, 2022, 49(6A): 363-369. https://doi.org/10.11896/jsjkx.210500044
[3] 罗俊仁, 张万鹏, 陆丽娜, 陈璟.
即时策略博弈在线对抗规划方法综述
Survey on Online Adversarial Planning for Real-time Strategy Game
计算机科学, 2022, 49(6): 287-296. https://doi.org/10.11896/jsjkx.210600168
[4] 张晓宇, 王彬, 安卫超, 阎婷, 相洁.
基于融合损失函数的3D U-Net++脑胶质瘤分割网络
Glioma Segmentation Network Based on 3D U-Net++ with Fusion Loss Function
计算机科学, 2021, 48(9): 187-193. https://doi.org/10.11896/jsjkx.200800099
[5] 黄颖琦, 陈红梅.
基于代价敏感卷积神经网络的非平衡问题混合方法
Cost-sensitive Convolutional Neural Network Based Hybrid Method for Imbalanced Data Classification
计算机科学, 2021, 48(9): 77-85. https://doi.org/10.11896/jsjkx.200900013
[6] 冯姣, 陆昶谕.
基于残差注意力网络的跨媒体检索方法
Cross Media Retrieval Method Based on Residual Attention Network
计算机科学, 2021, 48(6A): 122-126. https://doi.org/10.11896/jsjkx.201100026
[7] 段菲, 王慧敏, 张超.
面向数据表示的Cauchy非负矩阵分解
Cauchy Non-negative Matrix Factorization for Data Representation
计算机科学, 2021, 48(6): 96-102. https://doi.org/10.11896/jsjkx.200700195
[8] 石先让, 宋廷伦, 唐得志, 戴振泳.
一种新颖的单目视觉深度学习算法:H_SFPN
Novel Deep Learning Algorithm for Monocular Vision:H_SFPN
计算机科学, 2021, 48(4): 130-137. https://doi.org/10.11896/jsjkx.200400090
[9] 曲浩, 崔超然, 王萧萧, 苏雅茜, 韩晓晖, 尹义龙.
基于非均衡数据层次学习的案件案由预测方法
Hierarchical Learning on Unbalanced Data for Predicting Cause of Action
计算机科学, 2021, 48(12): 337-342. https://doi.org/10.11896/jsjkx.201100212
[10] 孟丽莎, 任坤, 范春奇, 黄泷.
基于密集卷积生成对抗网络的图像修复
Dense Convolution Generative Adversarial Networks Based Image Inpainting
计算机科学, 2020, 47(8): 202-207. https://doi.org/10.11896/jsjkx.190700017
[11] 景雨, 祁瑞华, 刘建鑫, 刘朝霞.
基于改进多尺度深度卷积网络的手势识别算法
Gesture Recognition Algorithm Based on Improved Multiscale Deep Convolutional Neural Network
计算机科学, 2020, 47(6): 180-183. https://doi.org/10.11896/jsjkx.200200030
[12] 王立华,杜明辉,梁亚玲.
基于角度特征的分类网络
Classification Net Based on Angular Feature
计算机科学, 2020, 47(2): 83-87. https://doi.org/10.11896/jsjkx.190500077
[13] 王丽星, 曹付元.
基于Huber损失的非负矩阵分解算法
Huber Loss Based Nonnegative Matrix Factorization Algorithm
计算机科学, 2020, 47(11): 80-87. https://doi.org/10.11896/jsjkx.190900144
[14] 鲁淑霞, 蔡莲香, 张罗幻.
基于零阶减小方差方法的鲁棒支持向量机
Robust SVM Based on Zeroth Order Variance Reduction
计算机科学, 2019, 46(11): 193-201. https://doi.org/10.11896/jsjkx.181001840
[15] 陶秉墨,鲁淑霞.
基于自适应随机梯度下降方法的非平衡数据分类
Adaptive Stochastic Gradient Descent for Imbalanced Data Classification
计算机科学, 2018, 45(6A): 487-492.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!