Computer Science

Computer Science ›› 2024, Vol. 51 ›› Issue (7): 327-336.doi: 10.11896/jsjkx.230500036

• Artificial Intelligence • Previous Articles     Next Articles

Deep-Init:Non Joint Initialization Method for Visual Inertial Odometry Based on Deep Learning

SHI Dianxi1,4, GAO Yunqi1,2, SONG Linna3, LIU Zhe3, ZHOU Chenlei4, CHEN Ying2   

  1. 1 Intelligent Game and Decision Lab(IGDL),Beijing 100091,China
    2 National Innovation Institute of Defense Technology,Beijing 100071,China
    3 College of Computer,National University of Defense Technology,Changsha 410073,China
    4 Tianjin(Binhai) Artificial Intelligence Innovation Center,Tianjin 300450,China
  • Received:2023-05-06 Revised:2023-10-15 Online:2024-07-15 Published:2024-07-10
  • About author:SHI Dianxi,born in 1966,Ph.D,professor,Ph.D supervisor.His main research interests include artificial intelligence,robot operating system,distributed computing,and cloud computing.
    CHEN Ying,born in 1985,Ph.D,assistant research fellow.Her main research interests include artificial intelligence algorithm and framework design and optimization.
  • Supported by:
    Science and Technology Innovation 2030 Major Project(2020AAA0104802) and National Natural Science Foundation of China(91948303).

PDF (PC)

357

Abstract: For a non-linear monocular VIO system,its initialization process is crucial,and the initialization result directly affects the accuracy of the state estimation during the whole system operation.To this end,this paper introduces a deep learning method into the initialization process of the monocular VIO system and proposes an efficient non-joint initialization method(referred to as Deep-Init).The core of this method is to use a deep neural network to accurately estimate the random error terms such as bias and noise of the gyroscope in the IMU,to obtain the key parameter in the initialization process,i.e.the bias of the gyroscope.At the same time,we loosely couple the IMU pre-integration to the SfM.The absolute scale,velocity and gravity vector are quickly recovered by position and rotation alignment using least squares,which are used as initial values to guide the non-linear tightly coupled optimization framework.The accuracy of the rotation estimates in the IMU is greatly increased due to the compensation of the gyroscope data by the deep neural network,which effectively improves the signal-to-noise ratio of the IMU data.This also reduces the number of least squares equation failures,further reducing the computational effort.Using the pre-integrated amount of gyroscope data with the error term removed to replace the rotation amount in the SfM and using the IMU rotation amount as the true value,not only avoids the errors associated with initializing inaccurate SfM values as the true value but slao effectively improves the accuracy of system state estimation.Moreover,it enables effective adaptation to scenarios where SfM estimation is poor,such as high-speed motion,drastic lighting changes and texture repetition.The validity of the proposed method is verified on the EuRoC dataset,and the experimental results show that the proposed Deep-Init initialization method achieves good results in terms of both accuracy and time consumption.

Key words: Visual-inertial odometry, Deep learning, Initialization, Inertial measurement unit

CLC Number: 

  • TP391.41
[1]MUR-ARTAL R,MONTIEL J M M,TARDOS J D.ORB-SLAM:a versatile and accurate monocular SLAM system[J].IEEE Transactions on Robotics,2015,31(5):1147-1163.
[2]MUR-ARTAL R,TARDÓS J D.Orb-slam2:An open-sourceslam system for monocular,stereo,and rgb-d cameras[J].IEEE Transactions on Robotics,2017,33(5):1255-1262.
[3]ENGEL J,KOLTUN V,CREMERS D.Direct sparse odometry[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,40(3):611-625.
[4]QIN T,LI P,SHEN S.Vins-mono:A robust and versatile mo-nocular visual-inertial state estimator[J].IEEE Transactions on Robotics,2018,34(4):1004-1020.
[5]GENEVA P,ECKENHOFF K,LEE W,et al.OpenVINS:A Research Platform for Visual-Inertial Estimation[C]//Proceedings of the IEEE International Conference on Robotics and Automation.IEEE,2020.
[6]QIN T,SHEN S.Robust initialization of monocular visual-inertial estimation on aerial robots[C]//2017 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS).IEEE,2017:4225-4232.
[7]CAMPOS C,MONTIEL J M M,TARDÓS J D.Fast and robust initialization for visual-inertial SLAM[C]//2019 International Conference on Robotics and Automation(ICRA).IEEE,2019:1288-1294.
[8]YANG Z,SHEN S.Monocular visual-inertial state estimation with online initialization and camera-IMU extrinsic calibration[J].IEEE Transactions on Automation Science and Engineering,2016,14(1):39-51.
[9]CAMPOS C,ELVIRA R,RODRÍGUEZ J J G,et al.ORB-SLAM3:An Accurate Open-Source Library for Visual,Visual-Inertial,and Multimap SLAM[J].IEEE Transactions on Robo-tics,2021,37(6):1874-1890.
[10]DOMÍNGUEZ-CONTI J,YIN J,ALAMI Y,et al.Visual-inertial slam initialization:A general linear formulation and a gravity-observing non-linear optimization[C]//2018 IEEE International Symposium on Mixed and Augmented Reality(ISMAR).IEEE,2018:37-45.
[11]MUR-ARTAL R,TARDÓS J D.Visual-inertial monocularSLAM with map reuse[J].IEEE Robotics and Automation Letters,2017,2(2):796-803.
[12]MUSTANIEMI J,KANNALA J,SÄRKKÄ S,et al.Inertial-based scale estimation for structure from motion on mobile devices[C]//2017 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS).IEEE,2017:4394-4401.
[13]SNAVELY N,SEITZ S M,SZELISKI R.Modeling the world from internet photo collections[J].International Journal of Computer Vision,2008,80(2):189-210.
[14]FORSTER C,CARLONE L,DELLAERT F,et al.On-manifold preintegration for real-time visual-inertial odometry[J].IEEE Transactions on Robotics,2016,33(1):1-21.
[15]MARTINELLI A.Closed-form solution of visual-inertial structure from motion[J].International Journal of Computer Vision,2014,106(2):138-152.
[16]KAISER J,MARTINELLI A,FONTANA F,et al.Simulta-neous state initialization and gyroscope bias calibration in visual inertial aided navigation[J].IEEE Robotics and Automation Letters,2016,2(1):18-25.
[17]HUANG W,LIU H.Online initialization and automatic camera-IMU extrinsic calibration for monocular visual-inertial SLAM[C]//2018 IEEE International Conference on Robotics and Automation(ICRA).IEEE,2018:5182-5189.
[18]CAMPOS C,MONTIEL J M M,TARDÓS J D.Inertial-only optimization for visual-inertial initialization[C]//2020 IEEE International Conference on Robotics and Automation(ICRA).IEEE,2020:51-57.
[19]GAO Y Q,SHI D X,LI R H,et al.Gyro-Net:IMU Gyroscopes Random Errors Compensation Method Based on Deep Learning[C]//IEEE Robotics and Automation Letters.
[20]ESFAHANI M A,WANG H,WU K,et al.OriNet:Robust 3-D orientation estimation with a single particular IMU[J].IEEE Robotics and Automation Letters,2019,5(2):399-406.
[21]BROSSARD M,BONNABEL S,BARRAU A.Denoising imugyroscopes with deep learning for open-loop attitude estimation[J].IEEE Robotics and Automation Letters,2020,5(3):4796-4803.
[22]LUPTON T,SUKKARIEH S.Visual-inertial-aided navigationfor high-dynamic motion in built environments without initial conditions[J].IEEE Transactions on Robotics,2011,28(1):61-76.
[23]HORAUD R,DORNAIKA F.Hand-eye calibration[J].The International Journal of Robotics Research,1995,14(3):195-210.
[1] YANG Heng, LIU Qinrang, FAN Wang, PEI Xue, WEI Shuai, WANG Xuan. Study on Deep Learning Automatic Scheduling Optimization Based on Feature Importance [J]. Computer Science, 2024, 51(7): 22-28.
[2] LI Jiaying, LIANG Yudong, LI Shaoji, ZHANG Kunpeng, ZHANG Chao. Study on Algorithm of Depth Image Super-resolution Guided by High-frequency Information ofColor Images [J]. Computer Science, 2024, 51(7): 197-205.
[3] FAN Yi, HU Tao, YI Peng. Host Anomaly Detection Framework Based on Multifaceted Information Fusion of SemanticFeatures for System Calls [J]. Computer Science, 2024, 51(7): 380-388.
[4] GAN Run, WEI Xianglin, WANG Chao, WANG Bin, WANG Min, FAN Jianhua. Backdoor Attack Method in Autoencoder End-to-End Communication System [J]. Computer Science, 2024, 51(7): 413-421.
[5] HUANG Haixin, CAI Mingqi, WANG Yuyao. Review of Point Cloud Semantic Segmentation Based on Graph Convolutional Neural Networks [J]. Computer Science, 2024, 51(6A): 230400196-7.
[6] WANG Yingjie, ZHANG Chengye, BAI Fengbo, WANG Zumin. Named Entity Recognition Approach of Judicial Documents Based on Transformer [J]. Computer Science, 2024, 51(6A): 230500164-9.
[7] LIANG Fang, XU Xuyao, ZHAO Kailong, ZHAO Xuanfeng, ZHANG Guijun. Remote Template Detection Algorithm and Its Application in Protein Structure Prediction [J]. Computer Science, 2024, 51(6A): 230600225-7.
[8] PENG Bo, LI Yaodong, GONG Xianfu, LI Hao. Method for Entity Relation Extraction Based on Heterogeneous Graph Neural Networks and TextSemantic Enhancement [J]. Computer Science, 2024, 51(6A): 230700071-5.
[9] ZHANG Tianchi, LIU Yuxuan. Research Progress of Underwater Image Processing Based on Deep Learning [J]. Computer Science, 2024, 51(6A): 230400107-12.
[10] WANG Guogang, DONG Zhihao. Lightweight Image Semantic Segmentation Based on Attention Mechanism and Densely AdjacentPrediction [J]. Computer Science, 2024, 51(6A): 230300204-8.
[11] ZHANG Le, YU Ying, GE Hao. Mural Inpainting Based on Fast Fourier Convolution and Feature Pruning Coordinate Attention [J]. Computer Science, 2024, 51(6A): 230400083-9.
[12] WU Yibo, HAO Yingguang, WANG Hongyu. Rice Defect Segmentation Based on Dual-stream Convolutional Neural Networks [J]. Computer Science, 2024, 51(6A): 230600107-8.
[13] HOU Linhao, LIU Fan. Remote Sensing Image Fusion Combining Multi-scale Convolution Blocks and Dense Convolution Blocks [J]. Computer Science, 2024, 51(6A): 230400110-6.
[14] HUANG Yuanhang, BIAN Shan, WANG Chuntao. Gaussian Enhancement Module for Reinforcing High-frequency Details in Camera ModelIdentification [J]. Computer Science, 2024, 51(6A): 230700125-5.
[15] SUN Yang, DING Jianwei, ZHANG Qi, WEI Huiwen, TIAN Bowen. Study on Super-resolution Image Reconstruction Using Residual Feature Aggregation NetworkBased on Attention Mechanism [J]. Computer Science, 2024, 51(6A): 230600039-6.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.