计算机科学 ›› 2025, Vol. 52 ›› Issue (5): 199-211.doi: 10.11896/jsjkx.240400172

• 计算机图形学&多媒体 • 上一篇    下一篇

基于深度位置感知Transformer的低重叠点云配准

孔煜1, 熊风光1,2,3, 张志强1, 申超凡1, 胡明月1   

  1. 1 中北大学计算机科学与技术学院 太原 030051
    2 机器视觉与虚拟现实山西省重点实验室 太原 030051
    3 山西省视觉信息处理及智能机器人工程研究中心 太原 030051
  • 收稿日期:2024-04-24 修回日期:2024-08-13 出版日期:2025-05-15 发布日期:2025-05-12
  • 通讯作者: 熊风光(hopenxfg@nuc.edu.cn)
  • 作者简介:(794102577@qq.com)
  • 基金资助:
    国家自然科学基金(62272426);山西省自然科学基金(202203021212138);山西省科技重大专项计划“揭榜挂帅”项目(202201150401021)

Low Overlap Point Cloud Registration Method Based on Deep Position-aware Transformer

KONG Yu1, XIONG Fengguang1,2,3, ZHANG Zhiqiang1, SHEN Chaofan1, HU Mingyue1   

  1. 1 School of Computer Science and Technology,North University of China,Taiyuan 030051,China
    2 Shanxi Provincial Key Laboratory of Machine Vision and Virtual Reality,Taiyuan 030051,China
    3 Shanxi Province's Vision Information Processing and Intelligent Robot Engineering Research Center,Taiyuan 030051,China
  • Received:2024-04-24 Revised:2024-08-13 Online:2025-05-15 Published:2025-05-12
  • About author:KONG Yu,born in 1999,postgraduate,is a member of CCF(No.N7917G).His main research interests include com-puter vision and so on.
    XIONG Fengguang,born in 1979,Ph.D,associate professor,master supervisor.His main research interests include computer vision and pattern recognition,virtual simulation and visualization and data visualization,etc.
  • Supported by:
    National Natural Science Foundation of China(62272426),Natural Science Foundation of Shanxi Province,China(202203021212138) and Science and Technology Major Special Plan “Reveal the List” Project of Shanxi Province,China(202201150401021).

摘要: 针对特征提取阶段忽视局部几何嵌入的融合,特征交互阶段低重叠点云对之间的位置感知信息呈现弱相关性导致难以提取更富有表现力的特征,以及对应生成阶段出现部分错误对应导致求解的变换矩阵存在偏差等问题,提出了一种基于深度位置感知Transformer(DeepPAT)的三维点云低重叠配准方法。首先,设计了融合局部几何信息的局部特征提取网络,用于提取点云多层次特征;然后,设计了基于深度位置感知的Transformer(PAT)模块,通过学习点云自身和跨帧的几何和深度空间位置信息,提取低重叠率的源点云和目标点云的相关特征和重叠信息,以便进行低重叠特征匹配;最后,设计了由特征相似性项调节的极大团算法来减轻长度一致性所带来的空间模糊性,从而过滤离群点。其可作为一种即插即用的估计模块代替RANSAC等传统鲁棒估计器。在室内3DMatch数据集和合成ModelNet数据集上进行评估,实验结果表明:在测试ModelNet数据集的旋转和平移均方根误差方面,DeepPAT分别将误差降低至3.994和0.005;在测试3DMatch和3DLoMatch基准的配准召回率方面,DeepPAT分别比现有方法高出至少4.3%和3.6%。

关键词: 低重叠率, 极大团, 局部特征提取, 深度位置感知, 局部到全局匹配

Abstract: In response to the issues such as neglecting the fusion of local geometric embeddings in the feature extraction stage,weak correlation in position-aware information between low overlap point cloud pairs in the feature interaction stage,making it difficult to extract more expressive features and deviation in the transformation solved due to some outlier correspondence in the correspondence generation stage,in this paper,a 3D point cloud low overlap registration method based on deep position-aware Transformer(DeepPAT) is proposed,which follows the local to global matching mechanism.A local feature extraction network based on local geometry information is proposed to extract multi-level features from point cloud.Then,a deep position-aware Transformer(DPAT) module is designed to extract the relevant features and overlap information between low overlap point cloud pairs by learning the geometry and spatial position information of the point cloud itself and across frames,so as to carry out low overlap point cloud matching.Finally,a maximal cliques algorithm adjusted by the feature similarity is designed to reduce the position ambiguity caused by the length consistency and eliminate the outlier correspondences.It can be used as a plug-and-play robust estimator to replace traditional robust estimators such as RANSAC and is fully implemented by Pytorch.Evaluating on the synthetic ModelNet dataset and indoor 3DMatch dataset,the experimental results show that DeepPAT reduces the rotation and translation root mean square error to 3.994 and 0.005 on ModelNet datasets,respectively,and DeepPAT outperformed existing methods by at least 4.3 percentage points and 3.6 percentage points in term of registration recall on 3DMatch and 3DLoMatch benchmarks,respectively.

Key words: Low overlap ratio, Maximal cliques, Local feature extraction, Deep position awareness, Local to global matching

中图分类号: 

  • TP391.41
[1]CHINMAY K,WAGLE K.Review of SLAM algorithms for indoor mobile robot with LIDAR and RGB-D camera technology[C]//Innovations in Electrical and Electronic Engineering:Proceedings of ICEEE 2020.2021:397-409.
[2]LI S T,LI J L,MENG Q Y,et al.Loop-closure detection algorithm based on point cloud histogram and vehicle positioning method.[J].Journal of Jilin University(Engineering and Technology Edition),2023,53(8):2395-2403.
[3]GUO Y,BENNAMOUN M,SOHEL F,et al.3D object recognition in cluttered scenes with local surface features:A survey[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2014,36(11):2270-2287.
[4]ZERMAS D,IZZAT I,PAPANIKOLOPOULOS N.Fast seg-mentation of 3d point clouds:A paradigm on lidar data for autonomous vehicle applications[C]//2017 IEEE International Conference on Robotics and Automation(ICRA).IEEE,2017:5067-5073.
[5]CAI H,KOU T T,YANG Y N,et al.Three-dimensional vehicle multi-target tracking based on trajectory optimization.[J].Journal of jilin university(engineering and technology edition),2024,54(8):2338-2347.
[6]MUSIALSKI P,WONKA P,ALIAGA D G,et al.A Survey of Urban Reconstruction[J].Computer Graphics Forum,2013,32(6):146-177.
[7]BESL P J,MCKAY H D.A method for registration of 3-Dshapes[J].IEEE Transactionson Pattern Analysis & Machine Intelligence,1992,14(2):239-256.
[8]YANG J,LI H,CAMPBELL D,et al.Go-ICP:A Globally Optimal Solution to 3D ICP Point-Set Registration[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2016,38(11):2241-2254.
[9]RUSU R B,BLODOW N,MARTON Z C,et al.Aligning point cloud views using persistent feature histograms[C]//2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.IEEE,2008:3384-3391.
[10]RUSU R B,BLODOW N,BEETZ M.Fast point feature histograms(FPFH) for 3D registration[C]//2009 IEEE Interna-tional Conference on Robotics and Automation.IEEE,2009:3212-3217.
[11]DENG H,BIRDAL T,ILIC S.Ppf-foldnet:Unsupervised learning of rotation invariant 3d local descriptors[C]//Proceedings of the European Conference on Computer Vision(ECCV).2018:602-618.
[12]ZHANG X,YANG J,ZHANG S,et al.3D registration withmaximal cliques[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023:17745-17754.
[13]FISCHLER M A,BOLLES R C.Random sample consensus:aparadigm for model fitting with applications to image analysis and automated cartography[J].Communications of the ACM,1981,24(6):381-395.
[14]CHOY C,PARK J,KOLTUN V.Fully convolutional geomet-ricfeatures[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019:8958-8966.
[15]BAI X,LUO Z,ZHOU L,et al.D3feat:Joint learning of dense detection and description of 3d local features[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:6359-6367.
[16]THOMAS H,QI C R,DESCHAUD J E,et al.Kpconv:Flexible and deformable convolution for point clouds[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019:6411-6420.
[17]HUANG S,GOJCIC Z,USVYATSOV M,et al.Predator:Reg-istration of 3d point clouds with low overlap[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:4267-4276.
[18]YU H,LI F,SALEH M,et al.Cofinet:Reliable coarse-to-finecorrespondences for robust pointcloud registration[J].Advances in Neural Information Processing Systems,2021,34:23872-23884.
[19]AO S,HU Q,YANG B,et al.Spinnet:Learning a general surface descriptor for 3d point cloud registration[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:11753-11762.
[20]WANG H,LIU Y,DONG Z,et al.You only hypothesize once:Point cloud registration with rotation-equivariant descriptors[C]//Proceedings of the 30th ACM International Conference on Multimedia.2022:1630-1641.
[21]YU H,QIN Z,HOU J,et al.Rotation-invariant transformer for point cloud matching[C]//Proce-edings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023:5384-5393.
[22]AOKI Y,GOFORTH H,SRIVATSAN R A,et al.Pointnetlk:Robust & efficient point cloud registration using pointnet[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:7163-7172.
[23]QI C R,SU H,MO K,et al.Pointnet:Deep learning on pointsets for 3d classification and segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:652-660.
[24]SARODE V,LI X,GOFORTH H,et al.Pcrnet:Point cloud registration network using pointnet encoding[J].arXiv:1908.07906,2019.
[25]XU H,LIU S,WANG G,et al.Omnet:Learning overlap** mask for partial-to-partial point cloud registration[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:3132-3141.
[26]WANG Y,SOLOMON J M.Deep closest point:Learning representations for point cloud registration[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019:3523-3532.
[27]WANG Y,SOLOMON J M.Prnet:Self-supervised learning for partial-to-partial registration[J].arXiv:1910.12240,2019.
[28]JANG E,GU S,POOLE B.Categorical reparameterization with gumbel-softmax[J].arXiv:1611.01144,2016.
[29]LI J,ZHANG C,XU Z,et al.Iterative distance-aware similarity matrix convolution with mutual-supervised point elimination for efficient point cloud registration[C]//Computer Vision-ECCV 2020:16th European Conference,Glasgow,UK,August 23-28,2020,Proceedings,Part XXIV 16.Springer International Publishing,2020:378-394.
[30]CHEN Z,CHEN H,GONG L,et al.UTOPIC:Uncertainty-aware Overlap Prediction Network for Partial Point Cloud Registration[J].Computer Graphics Forum,2022,41(7):87-98.
[31]FU K,LIU S,LUO X,et al.Robust point cloud registrationframework based on deep graph matching[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:8893-8902.
[32]YEW Z J,LEE G H.Regtr:End-to-end point cloud correspondenceswith transformers[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:6677-6686.
[33]QIN Z,YU H,WANG C,et al.Geometric transformer for fast and robust point cloud registration[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:11143-11152.
[34]YU H,HOU J,QIN Z,et al.Riga:Rotation-invariant and globally-awaredescriptors for point cloud registration[J].arXiv:2209.13252,2022.
[35]GOJCIC Z,ZHOU C,WEGNER J D,et al.The perfect match:3d point cloud matching with smoothed densities[C]//Procee-dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:5545-5554.
[36]ZHOU Q Y,PARK J,KOLTUN V.Fast global registration[C]//Computer Vision-ECCV 2016:14th European Confe-rence,Amsterdam,The Netherlands,October 11-14,2016,Proceedings,Part II 14.Springer International Publishing,2016:766-782.
[37]YUAN W,ECKART B,KIM K,et al.Deepgmr:Learning latent gaussian mixture models for registration[C]//Computer Vision-ECCV 2020:16th European Conference,Glasgow,UK,August 23-28,2020,Proceedings,Part V 16.Springer International Publishing,2020:733-750.
[38]LI X,PONTES J K,LUCEY S.Pointnetlk revisited[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:12763-12772.
[39]BAUER D,PATTEN T,VINCZE M.Reagent:Point cloud registration using imitation and reinforcement learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:14586-14594.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!