Computer Science ›› 2021, Vol. 48 ›› Issue (8): 157-161.doi: 10.11896/jsjkx.200700134

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Monocular Visual Odometer Based on Deep Learning SuperGlue Algorithm

LIU Shuai1, RUI Ting2, HU Yu-cheng1, YANG Cheng-song2, WANG Dong2   

  1. 1 School of Graduate,PLA Army Engineering University,Nanjing 210000,China;
    2 School of Field Engineering,PLA Army Engineering University,Nanjing 210000,China
  • Received:2020-07-21 Revised:2020-08-25 Published:2021-08-10
  • About author:LIU Shuai,born in 1994,postgraduate.His main research interests include computer vision machine learning and SLAM applications.(15737960205@163.com)RUI Ting,born in 1972,Ph.D,professor,master's advisor.His main research interests include image proces-sing,pattern recognition and artificial intelligence.
  • Supported by:
    National Key Research and Development Program of China(2016YFC0802904).

Abstract: Aiming at the visual odometer of feature point method,the change of illumination and view angle could lead to the instability of feature point extraction,which affects the accuracy of camera pose estimation,a monocular vision odometer modeling method based on deep learning SuperGlue matching algorithm is proposed.Firstly,the feature points are obtained by SuperPoint detector,and the resulting feature points are encoded to obtainvectors containing the coordinates and descriptors of the feature points.Then the more representative descriptors are generated by attentional GNN network.We useSinkhorn algorithm to solve the optimal score distribution matrix.Finally,according to the optimal feature matching,the camera pose is restored,and the ca-mera pose is optimized by using the minimum projection error equation.Experiments show that the proposed algorithm is not only more robust to view angle and light change than the visual odometer based on ORB or SIFT,without back-end optimization,but also the accuracy of absolute trajectory error and relative pose error is greatly improved,thus the feasibility and superiority of the deep learning based SuperGlue matching algorithm in visual slam are further verified.

Key words: Deep learning, Feature matching, GNN, SuperGlue, Visual odometer

CLC Number: 

  • TP391.9
[1]LU Y,SONG D.Visual navigation using heterogeneous land-marks and unsupervised .geo-metric constraints [J].IEEE Transactions on Robotics,2015,31(3):736-749.
[2]CHEN J W,BAUTEMBACH D,IZADI S.Scalable real-timevolumetric sur-face reconstruction[J].ACM Transactions on Graphics,2013,32(4):1-16.
[3]ABATE A F,NAPPI M,RICCIO D,et al.2D and 3D face recognition:a survey [J].Pattern Recognition Letters,2007,28(14):1885-1906.
[4]JIA K,CHAN T H,ZENG Z,et al.ROML:A robust feature correspondence approach for matching objects in a set of images [J].International Journal of Computer Vision,2016,117(2):173-197.
[5]SONG X,ZHAO X,HU H W,et al.EdgeStereo:a context integrated residual pyra-mid network for stereo matching[C]//Proc of Asian Conference on Computer Vision.2018.
[6]YI K,TRULLS E,LEPETIT V,et al.LIFT:Learned invariant feature transform[C]//Computer Vision(ECCV 2016).2016: 467-483.
[7]MIHAI D,IGNACIO R,TOMAS P et al.D2-Net:A trainable CNN for joint detection and description of local features[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).2019;8092-8101.
[8]DANIEL D,TOMASZ M,A-NDREW R.Self-improvingvisualodometry[J].arXiv:1812.03245,2018.
[9]DANIEL D,TOMASZ M,ANDREW R.SuperPoint:Self-supervised interest point detection and description[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops.2018:224-236.
[10]BALNTANS V,LENC K,VEDALDI A,et al.HPatches:Abenchmark and evaluation of handcrafted and learned local descriptors[C]//Proceedings of the IEEE Conference on Compu-ter Vision and Pattern Recognition (CVPR).2017:5173-5182.
[11]PAULE S,DANIEL D,TOMASZ M,et al.SuperGlue:Learning Feature Matching With Graph Neural Networks[C]//Procee-dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).2020:4938-4947.
[12]DANIEL D,TOMASZ M,A-NDREW R.Deep image homography estimation[J].arXiv:1606.03798,2016.2.
[13]JUHO L,YOONHO L,JUNGTAEK K,et al.Set Transformer:A framework for attention-based permutation-invariant neural networks[C]//Proceedings of the 36th International Conference on Machine Learning(PMLR 97).2019:3744-3753.
[14]LUO Z X,SHEN T W,ZHOUL,et al.ContextDesc:Local descriptor augmentation with cross-modality context[C]//Proceedings of theIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).2019:2527-2536.
[1] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[2] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[3] HU An-xiang, YIN Xiao-kang, ZHU Xiao-ya, LIU Sheng-li. Strcmp-like Function Identification Method Based on Data Flow Feature Matching [J]. Computer Science, 2022, 49(9): 326-332.
[4] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[5] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[6] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[7] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[8] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[9] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[10] ZHOU Hui, SHI Hao-chen, TU Yao-feng, HUANG Sheng-jun. Robust Deep Neural Network Learning Based on Active Sampling [J]. Computer Science, 2022, 49(7): 164-169.
[11] SU Dan-ning, CAO Gui-tao, WANG Yan-nan, WANG Hong, REN He. Survey of Deep Learning for Radar Emitter Identification Based on Small Sample [J]. Computer Science, 2022, 49(7): 226-235.
[12] HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[13] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[14] WANG Jun-feng, LIU Fan, YANG Sai, LYU Tan-yue, CHEN Zhi-yu, XU Feng. Dam Crack Detection Based on Multi-source Transfer Learning [J]. Computer Science, 2022, 49(6A): 319-324.
[15] CHU Yu-chun, GONG Hang, Wang Xue-fang, LIU Pei-shun. Study on Knowledge Distillation of Target Detection Algorithm Based on YOLOv4 [J]. Computer Science, 2022, 49(6A): 337-344.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!