计算机科学 ›› 2020, Vol. 47 ›› Issue (9): 198-203.doi: 10.11896/jsjkx.191000040

• 人工智能 • 上一篇    下一篇

动态环境下的语义地图构建

齐少华, 徐和根, 万友文, 付豪   

  1. 同济大学电子与信息工程学院 上海201804
  • 收稿日期:2019-10-09 发布日期:2020-09-10
  • 通讯作者: 徐和根(xuhegen@tongji.edu.cn)
  • 作者简介:2316187741@qq.com

Construction of Semantic Mapping in Dynamic Environments

QI Shao-hua, XU He-gen, WAN You-wen, FU Hao   

  1. College of Electronics and Information Engineering,Tongji University,Shanghai 201804,China
  • Received:2019-10-09 Published:2020-09-10
  • About author:QI Shao-hua,born in 1995,postgra-duate.His main research interests include visual SLAM,deep learning and augmented reality.
    XU He-gen,born in 1972,Ph.D,asso-ciate professor.His main research inte-rests include image processing,machine vision,pattern recognition and robot.

摘要: 三维语义地图在移动机器人的导航、路径规划、智能抓取、人机交互等任务中有着关键的作用,因此如何实时地构建三维语义地图尤为重要。当前同时定位和地图构建(Simultaneous Localization And Mapping,SLAM)算法已经可以达到较高的定位和制图精度,但是在动态环境下如何通过剔除动态物体来获得较高的定位精度,以及理解周围场景中存在的物体及其位置信息等问题没有得到很好的解决。在此,文中提出了一种可在动态环境下构建语义地图的算法。该算法在ORB-SLAM2上进行改进,在跟踪线程中加入动静点检测算法来剔除检测为动点的特征点,提高了动态环境下的定位精度;添加目标检测线程对关键图像进行目标检测,在地图构建线程中构建Octo-Map地图,同时根据检测结果构建3D目标数据库。为了证明该算法的可行性,以实验室为测试环境,分别进行了目标检测、动态点检测、三维目标信息获取和动态环境下语义地图构建的实验。在目标检测实验中,训练了速度和精度较高的目标检测网络——mobilenet-v2-ssdlite,检测速度可以达到7帧/秒,基本可以实现实时检测。在动态点检测中,采用光流法剔除动态点,处理速度为16.5帧/秒。文中创建了数据集来评测算法性能,相比原版ORB-SLAM2算法,结合光流法后的算法的定位精度提高了5倍;在三维目标信息获取上,采用了基于深度滤波和基于点云分割两种方法,结果表明后者的3D目标获取更为精确。最后,对整个实验室进行动态环境下的语义地图构建,构建Octo-Map稠密地图,根据检测结果构建3D目标数据库,并将目标尺寸和位置的检测值与真实值进行对比,误差均在5厘米以内。实验结果表明所提算法具有较高的精度和实时性。

关键词: 动态点检测, 目标检测, 视觉SLAM, 语义地图构建

Abstract: Three-dimensional semantic maps play a key role in tasks such as robot navigation,path planning,intelligent grasping and human-computer interaction.So how to construct 3D semantic maps in real time is especially important.The current SLAM (simultaneous localization and mapping) algorithm can achieve higher positioning and mapping accuracy.However,how to eliminate dynamic objects to obtain higher positioning accuracy in a dynamic environment,and to understand the existence of objects and their location information in the surrounding scenes are still not well solved.This paper presents an algorithm for constructing semantic maps in a dynamic environment.This algorithm is improved on ORB-SLAM2.The dynamic and static point detection algorithm is added to the tracking thread to eliminate the feature points detected as dynamic feature points,which improves the positioning accuracy in dynamic environment.Object detection threads are added to detect key images.The mapping threads are added with the Octo-Map dense map construction.At the same time,the 3D object database is constructed according to the detection results.In order to prove the feasibility of the algorithm,the laboratory is used as the test environment,and the object detection,dynamic point detection,3D target information acquisition,and semantic map construction experiments in the dynamic environment are performed.In the object detection experiment,a high-speed and high-precision object detection network,mobilenet-v2-ssdlite,is trained,which can reach a detection speed of 7 frames/s,which can basically achieve real-time detection.In dynamic point detection,the optical flow method was used to eliminate dynamic point,processing speed is 16.5 frames/s.And this paper creates a data set to evaluate the performance of the algorithm.Compared with the original ORB-SLAM2 algorithm,the positioning accuracy is improved by 5 times after combining with the optical flow method.For the acquisition of three-dimensional object information,two methods based on depth filtering and point cloud segmentation are adopted.The results show that the latter’s 3D object acquisition is more accurate.Finally,the entire laboratory is constructed with a semantic map in a dynamic environment,an Octo-Map dense map is constructed,and a 3D object database is constructed based on the detection results.The detected values of the object size and position are compared with the true values,and the errors are within 5cm.The results show that the proposed algorithm has high accuracy and real-time performance.

Key words: Dynamic point detection, Object detection, Semantic mapping, Visual SLAM

中图分类号: 

  • TP249
[1] LIU W,ANGUELOV D,ERHAN D,et al.SSD:single shotmultibox detector[C]//European Conference on Computer Vision.Berlin,German:Springer,2016:21-37.
[2] REDMON J,DIVVALA S,GIRSHICK R,et al.You only look once:Unified,real-time object detection[J].arXiv:1506.02640 v4,2015.
[3] REN S,HE K,GIRSHICK R,et al.Faster R-CNN:Towards Real-Time Object Detection with Region Proposal Networks[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2015,39(6):1137-1149.
[4] HOWARD A,ZHU M,CHEN B,et al.Mobilenets:Efficientconvolutional neural networks for mobile vision applications[J].arXiv:1704.04861,2017.
[5] SANDLER M,HOWARD A,ZHU M L,et al.MobileNetV2:invertedresiduals and linear bottlenecks [J].arXiv:1801.04381[cs.CV],2018.
[6] MUR-ARTAL R,MONTIEL J M M,TARDOS J D.ORB-SLAM:a versatile and accurate monocular SLAM system [J].IEEE Transactions on Robotics,2015,31(5):1147-1163.
[7] MUR-ARTAL R,TARDOS J D.ORB-SLAM2:an Open-Source SLAM system for monocular stereo and RGB-D cameras [J].IEEE Transactions on Robotics,2017,33(5):1255-1262.
[8] BERTA B,FACIL J M,JAVIER C,et al.DynaSLAM:Track-ing,Mapping and Inpainting in Dynamic Scenes[J].arXiv:1806.05620v1,2018.
[9] YU C,LIU Z,LIU X,et al.DS-SLAM:A Semantic VisualSLAM towards Dynamic Environments[J].arXiv:1809.08379v2,2018.
[10] SALAS-MORENO R F,NEWCOMBE R A,STRASDAT H,et al.SLAM++:Simultaneous localisation and mapping at the level of objects[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,USA:IEEE,2013:1352-1359.
[11] HERMANS A,FLOROS G,LEIBE B.Dense 3D semantic mapping of indoor scenes from RGB-D images[C]//IEEE International Conference on Robotics and Automation.Piscataway,USA:IEEE,2014:2631-2638.
[12] CONCHA A,CIVERA J.DPPTAM:Dense piecewise planartracking and mapping from a monocular sequence[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems.Piscat- away,USA:IEEE,2015:5686-5693.
[13] TATENO K,TOMBARI F,LAINA I,et al.CNN-SLAM:Realtime dense monocular SLAM with learned depth prediction[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,USA:IEEE,2017:6565-6574.
[14] MCCORMAC J,HANDA A,DAVISON A,et al.Semantic Fu-sion Dense 3D semantic mapping with convolutional neural networks[C]//IEEE International Conference on Robotics and Automation.Piscataway,USA:IEEE,2017:4628-4635.
[15] JIANG W T.Construction of dense semantic map of large-scale road[D].Hangzhou:Zhejiang University,2016.
[16] YU J S,WU H,TIAN G H,et al.Semantic database design and semantic map construction of robots based on the cloud[J].Robot,2016,38(4):410-419.
[17] XING Q W.Construction of robot semantic map under unknown environment[D].Changchun:Northeast Normal University,2017.
[18] HAO W,GUOHUI T,PENG D,et al.Characterization of large-scale unknown environmental information based on RFID technology[J].Journal of Central South University(Natural Science Edition),2013(S1):166-170.
[19] ZHAO Z,CHEN X.Building 3D semantic maps for mobile robots using RGB-D camera[J].Intelligent Service Robotics,2016,9(4):1-13.
[20] HORNUNG A,WURM K M,BENNEWITZ M,et al.OctoMap:an efficient probabilistic 3D mapping framework based on octrees[J].Autonomous Robots,2013,34(3):189-206.
[21] JÜRGEN S,ENGELHARD N,ENDRES F,et al.A benchmark for the evaluation of RGB-D SLAM systems[C]//2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.IEEE,2012.
[1] 刘冬梅, 徐洋, 吴泽彬, 刘倩, 宋斌, 韦志辉.
基于边框距离度量的增量目标检测方法
Incremental Object Detection Method Based on Border Distance Measurement
计算机科学, 2022, 49(8): 136-142. https://doi.org/10.11896/jsjkx.220100132
[2] 王灿, 刘永坚, 解庆, 马艳春.
基于软标签和样本权重优化的Anchor Free目标检测算法
Anchor Free Object Detection Algorithm Based on Soft Label and Sample Weight Optimization
计算机科学, 2022, 49(8): 157-164. https://doi.org/10.11896/jsjkx.210600240
[3] 祝文韬, 兰先超, 罗唤霖, 岳彬, 汪洋.
改进Faster R-CNN的光学遥感飞机目标检测
Remote Sensing Aircraft Target Detection Based on Improved Faster R-CNN
计算机科学, 2022, 49(6A): 378-383. https://doi.org/10.11896/jsjkx.210300121
[4] 马宾, 付永康, 王春鹏, 李健, 王玉立.
基于GDIoU损失函数的YOLOv4绝缘子高效定位算法
High Performance Insulators Location Scheme Based on YOLOv4 with GDIoU Loss Function
计算机科学, 2022, 49(6A): 412-417. https://doi.org/10.11896/jsjkx.210600089
[5] 陈永平, 朱建清, 谢懿, 吴含笑, 曾焕强.
基于外接圆半径差损失的实时安全帽检测算法
Real-time Helmet Detection Algorithm Based on Circumcircle Radius Difference Loss
计算机科学, 2022, 49(6A): 424-428. https://doi.org/10.11896/jsjkx.220100252
[6] 陈佳舟, 赵熠波, 徐阳辉, 马骥, 金灵枫, 秦绪佳.
三维城市场景中的小物体检测
Small Object Detection in 3D Urban Scenes
计算机科学, 2022, 49(6): 238-244. https://doi.org/10.11896/jsjkx.210400174
[7] 胡伏原, 万新军, 沈鸣飞, 徐江浪, 姚睿, 陶重犇.
深度卷积神经网络图像实例分割方法研究进展
Survey Progress on Image Instance Segmentation Methods of Deep Convolutional Neural Network
计算机科学, 2022, 49(5): 10-24. https://doi.org/10.11896/jsjkx.210200038
[8] 徐涛, 陈奕仁, 吕宗磊.
基于改进YOLOv3的机坪工作人员反光背心检测研究
Study on Reflective Vest Detection for Apron Workers Based on Improved YOLOv3 Algorithm
计算机科学, 2022, 49(4): 239-246. https://doi.org/10.11896/jsjkx.210200119
[9] 张侣, 周博文, 吴亮红.
基于改进卷积注意力模块与残差结构的SSD网络
SSD Network Based on Improved Convolutional Attention Module and Residual Structure
计算机科学, 2022, 49(3): 211-217. https://doi.org/10.11896/jsjkx.201200019
[10] 赫晓慧, 邱芳冰, 程淅杰, 田智慧, 周广胜.
基于边缘特征融合的高分影像建筑物目标检测
High-resolution Image Building Target Detection Based on Edge Feature Fusion
计算机科学, 2021, 48(9): 140-145. https://doi.org/10.11896/jsjkx.200800002
[11] 袁磊, 刘紫燕, 朱明成, 马珊珊, 陈霖周廷.
融合改进密集连接和分布排序损失的遥感图像检测
Improved YOLOv3 Remote Sensing Target Detection Based on Improved Dense Connection and Distributional Ranking Loss
计算机科学, 2021, 48(9): 168-173. https://doi.org/10.11896/jsjkx.200800001
[12] 龚浩田, 张萌.
基于关键点检测的无锚框轻量级目标检测算法
Lightweight Anchor-free Object Detection Algorithm Based on Keypoint Detection
计算机科学, 2021, 48(8): 106-110. https://doi.org/10.11896/jsjkx.200700161
[13] 李琳, 刘学亮, 赵烨, 纪平.
结合乐高滤波器和SSD的低光照图像融合检测方法
Low Light Image Fusion Detection Method Based on Lego Filter and SSD
计算机科学, 2021, 48(7): 213-218. https://doi.org/10.11896/jsjkx.200800127
[14] 辛元雪, 史朋飞, 薛瑞阳.
基于区域提取与改进 LBP 特征的运动目标检测
Moving Object Detection Based on Region Extraction and Improved LBP Features
计算机科学, 2021, 48(7): 233-237. https://doi.org/10.11896/jsjkx.200600131
[15] 张曼, 李杰, 朱新忠, 沈霁, 成昊天.
基于改进DCGAN算法的遥感数据集增广方法
Augmentation Technology of Remote Sensing Dataset Based on Improved DCGAN Algorithm
计算机科学, 2021, 48(6A): 80-84. https://doi.org/10.11896/jsjkx.200700185
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!