计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (7): 327-336.doi: 10.11896/jsjkx.230500036

• 人工智能 • 上一篇    下一篇

Deep-Init:基于深度学习的视觉惯性里程计非联合初始化方法

史殿习1,4, 高云琦1,2, 宋林娜3, 刘哲3, 周晨磊4, 陈莹2   

  1. 1 智能博弈与决策实验室 北京 100091
    2 国防科技创新研究院 北京 100071
    3 国防科技大学计算机学院 长沙 410073
    4 天津(滨海)人工智能创新中心 天津 300457
  • 收稿日期:2023-05-06 修回日期:2023-10-15 出版日期:2024-07-15 发布日期:2024-07-10
  • 通讯作者: 陈莹(selina.ychen@foxmail.com)
  • 作者简介:(dxshi@nudt.edu.cn)
  • 基金资助:
    科技部科技创新2030-重大项目(2020AAA0104802);自然科学基金(91948303)

Deep-Init:Non Joint Initialization Method for Visual Inertial Odometry Based on Deep Learning

SHI Dianxi1,4, GAO Yunqi1,2, SONG Linna3, LIU Zhe3, ZHOU Chenlei4, CHEN Ying2   

  1. 1 Intelligent Game and Decision Lab(IGDL),Beijing 100091,China
    2 National Innovation Institute of Defense Technology,Beijing 100071,China
    3 College of Computer,National University of Defense Technology,Changsha 410073,China
    4 Tianjin(Binhai) Artificial Intelligence Innovation Center,Tianjin 300450,China
  • Received:2023-05-06 Revised:2023-10-15 Online:2024-07-15 Published:2024-07-10
  • About author:SHI Dianxi,born in 1966,Ph.D,professor,Ph.D supervisor.His main research interests include artificial intelligence,robot operating system,distributed computing,and cloud computing.
    CHEN Ying,born in 1985,Ph.D,assistant research fellow.Her main research interests include artificial intelligence algorithm and framework design and optimization.
  • Supported by:
    Science and Technology Innovation 2030 Major Project(2020AAA0104802) and National Natural Science Foundation of China(91948303).

PDF (PC)

360

摘要： 对于非线性的单目VIO系统来说,其初始化过程至关重要,初始化结果的好坏直接影响整个系统运行过程中状态估计的精度。为此,将深度学习方法引入单目VIO系统的初始化过程中,提出了一种高效的非联合初始化方法(简称Deep-Init),其核心是使用深度神经网络对IMU中陀螺仪的偏置和噪声等随机误差项进行准确估计,得到初始化过程中的关键参数,即陀螺仪的bias;同时,将IMU预积分与SfM进行松耦合,通过位置和旋转对齐,使用最小二乘法对绝对尺度、速度以及重力矢量等进行快速恢复,并将其作为初始值来引导非线性紧密耦合的优化框架。由于深度神经网络对陀螺仪数据进行补偿,从而大大提高了IMU中旋转估计量的准确性,有效提高了IMU数据的信噪比,同时减少了最小二乘方程失效的次数,因此进一步减少了计算量。使用去除误差项的陀螺仪数据的预积分量替换SfM中的旋转量,将IMU的旋转量作为真值,不仅避免了将不准确的SfM值作为真值进行初始化时所带来的误差,有效提升了系统状态估计的精度,而且能够有效地适应高速运动、光照变换剧烈和纹理重复等SfM估计效果差的场景。在EuRoC数据集上,对所提方法的有效性了进行实验验证,实验结果表明,所提出的初始化方法Deep-Init无论是精度还是耗时均取得了良好的效果。

关键词: 视觉惯性里程计, 深度学习, 初始化, 惯性测量单元

Abstract: For a non-linear monocular VIO system,its initialization process is crucial,and the initialization result directly affects the accuracy of the state estimation during the whole system operation.To this end,this paper introduces a deep learning method into the initialization process of the monocular VIO system and proposes an efficient non-joint initialization method(referred to as Deep-Init).The core of this method is to use a deep neural network to accurately estimate the random error terms such as bias and noise of the gyroscope in the IMU,to obtain the key parameter in the initialization process,i.e.the bias of the gyroscope.At the same time,we loosely couple the IMU pre-integration to the SfM.The absolute scale,velocity and gravity vector are quickly recovered by position and rotation alignment using least squares,which are used as initial values to guide the non-linear tightly coupled optimization framework.The accuracy of the rotation estimates in the IMU is greatly increased due to the compensation of the gyroscope data by the deep neural network,which effectively improves the signal-to-noise ratio of the IMU data.This also reduces the number of least squares equation failures,further reducing the computational effort.Using the pre-integrated amount of gyroscope data with the error term removed to replace the rotation amount in the SfM and using the IMU rotation amount as the true value,not only avoids the errors associated with initializing inaccurate SfM values as the true value but slao effectively improves the accuracy of system state estimation.Moreover,it enables effective adaptation to scenarios where SfM estimation is poor,such as high-speed motion,drastic lighting changes and texture repetition.The validity of the proposed method is verified on the EuRoC dataset,and the experimental results show that the proposed Deep-Init initialization method achieves good results in terms of both accuracy and time consumption.

Key words: Visual-inertial odometry, Deep learning, Initialization, Inertial measurement unit

中图分类号: 

  • TP391.41
[1]MUR-ARTAL R,MONTIEL J M M,TARDOS J D.ORB-SLAM:a versatile and accurate monocular SLAM system[J].IEEE Transactions on Robotics,2015,31(5):1147-1163.
[2]MUR-ARTAL R,TARDÓS J D.Orb-slam2:An open-sourceslam system for monocular,stereo,and rgb-d cameras[J].IEEE Transactions on Robotics,2017,33(5):1255-1262.
[3]ENGEL J,KOLTUN V,CREMERS D.Direct sparse odometry[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,40(3):611-625.
[4]QIN T,LI P,SHEN S.Vins-mono:A robust and versatile mo-nocular visual-inertial state estimator[J].IEEE Transactions on Robotics,2018,34(4):1004-1020.
[5]GENEVA P,ECKENHOFF K,LEE W,et al.OpenVINS:A Research Platform for Visual-Inertial Estimation[C]//Proceedings of the IEEE International Conference on Robotics and Automation.IEEE,2020.
[6]QIN T,SHEN S.Robust initialization of monocular visual-inertial estimation on aerial robots[C]//2017 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS).IEEE,2017:4225-4232.
[7]CAMPOS C,MONTIEL J M M,TARDÓS J D.Fast and robust initialization for visual-inertial SLAM[C]//2019 International Conference on Robotics and Automation(ICRA).IEEE,2019:1288-1294.
[8]YANG Z,SHEN S.Monocular visual-inertial state estimation with online initialization and camera-IMU extrinsic calibration[J].IEEE Transactions on Automation Science and Engineering,2016,14(1):39-51.
[9]CAMPOS C,ELVIRA R,RODRÍGUEZ J J G,et al.ORB-SLAM3:An Accurate Open-Source Library for Visual,Visual-Inertial,and Multimap SLAM[J].IEEE Transactions on Robo-tics,2021,37(6):1874-1890.
[10]DOMÍNGUEZ-CONTI J,YIN J,ALAMI Y,et al.Visual-inertial slam initialization:A general linear formulation and a gravity-observing non-linear optimization[C]//2018 IEEE International Symposium on Mixed and Augmented Reality(ISMAR).IEEE,2018:37-45.
[11]MUR-ARTAL R,TARDÓS J D.Visual-inertial monocularSLAM with map reuse[J].IEEE Robotics and Automation Letters,2017,2(2):796-803.
[12]MUSTANIEMI J,KANNALA J,SÄRKKÄ S,et al.Inertial-based scale estimation for structure from motion on mobile devices[C]//2017 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS).IEEE,2017:4394-4401.
[13]SNAVELY N,SEITZ S M,SZELISKI R.Modeling the world from internet photo collections[J].International Journal of Computer Vision,2008,80(2):189-210.
[14]FORSTER C,CARLONE L,DELLAERT F,et al.On-manifold preintegration for real-time visual-inertial odometry[J].IEEE Transactions on Robotics,2016,33(1):1-21.
[15]MARTINELLI A.Closed-form solution of visual-inertial structure from motion[J].International Journal of Computer Vision,2014,106(2):138-152.
[16]KAISER J,MARTINELLI A,FONTANA F,et al.Simulta-neous state initialization and gyroscope bias calibration in visual inertial aided navigation[J].IEEE Robotics and Automation Letters,2016,2(1):18-25.
[17]HUANG W,LIU H.Online initialization and automatic camera-IMU extrinsic calibration for monocular visual-inertial SLAM[C]//2018 IEEE International Conference on Robotics and Automation(ICRA).IEEE,2018:5182-5189.
[18]CAMPOS C,MONTIEL J M M,TARDÓS J D.Inertial-only optimization for visual-inertial initialization[C]//2020 IEEE International Conference on Robotics and Automation(ICRA).IEEE,2020:51-57.
[19]GAO Y Q,SHI D X,LI R H,et al.Gyro-Net:IMU Gyroscopes Random Errors Compensation Method Based on Deep Learning[C]//IEEE Robotics and Automation Letters.
[20]ESFAHANI M A,WANG H,WU K,et al.OriNet:Robust 3-D orientation estimation with a single particular IMU[J].IEEE Robotics and Automation Letters,2019,5(2):399-406.
[21]BROSSARD M,BONNABEL S,BARRAU A.Denoising imugyroscopes with deep learning for open-loop attitude estimation[J].IEEE Robotics and Automation Letters,2020,5(3):4796-4803.
[22]LUPTON T,SUKKARIEH S.Visual-inertial-aided navigationfor high-dynamic motion in built environments without initial conditions[J].IEEE Transactions on Robotics,2011,28(1):61-76.
[23]HORAUD R,DORNAIKA F.Hand-eye calibration[J].The International Journal of Robotics Research,1995,14(3):195-210.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发