Computer Science ›› 2023, Vol. 50 ›› Issue (1): 87-97.doi: 10.11896/jsjkx.211000118

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Viewpoint-tolerant Scene Recognition Based on Segmentation of Sparse Point Cloud

HE Xionghui1, TAN Jiefu1, LIU Zhe1, XUE Chao3, YANG Shaowu1, ZHANG Yongjun2   

  1. 1 College of Computer,National University of Defense Technology,Changsha 410073,China
    2 Artificial Intelligence Research Center,Defense Innovation Institute,Beijing 100166,China
    3 Tianjin(Binhai) Artificial Intelligence Innovation Center,Tianjing 300457,China
  • Received:2021-10-15 Revised:2022-04-13 Online:2023-01-15 Published:2023-01-09
  • About author:HE Xionghui,born in 1996,postgra-duate.His main research interests include place recognition in simultaneous localization and mapping.
    ZHANG Yongjun,born in 1966,Ph.D,professor.His main research interests include artificial intelligence,multi-agent cooperation,machine learning and feature recognition.
  • Supported by:
    National Natural Science Foundation of China(91948303) and Science and Technology Commission of Tianjin Binhai New Area(BHXQKJXM-PT-RGZNJMZX-2019001).

Abstract: In autonomous robot navigation,simultaneous localization and mapping is responsible for perceiving the surrounding environment and positioning itself,providing perceptual support for subsequent advanced tasks.Scene recognition,as a key mo-dule,can help the robot perceive the surrounding environment more accurately.It can correct the accumulated error caused by sensor error by identifying whether the current observation and the previous observation belong to the same scene.Existing me-thods mainly focus on scene recognition under the stable viewpoint,and judge whether two observations belong to the same scene based on the visual similarity between them.However,when the observation angle changes,there may be large visual differences in observations of the same scene,which may make the observations only partially similar,and this will lead to the failure of traditional methods.Therefore,a scene recognition method based on sparse point cloud segmentation is proposed.It divides the scene to solve local similar problems,and combines visual information and geometric information to achieve accurate scene description and ma-tching.So that the robot can recognize the same scene observation under different perspectives,which supports the loop detection for a single robot or the map fusion for multi-robot.This method divides each observation into several parts based on sparse point cloud segmentation.The segmentation result is invariant to the perspective,and each segment is extracted with a local bag of words vector and a β angle histogram to accurately describe its scene content.The former contains the visual semantic information of the scene.The latter contains the geometric structure information of the scene.Then,based on the segment,the same parts between observations are matched,the different parts are discarded to achieve accurate scene content matching and improve the success rate of place recognition.Finally,results on the public dataset show that this method outperforms the mainstream method bag of words in both stable and changing perspectives.

Key words: Visual scene recognition, Segmentation, Sparse point cloud, Simultaneous localization and mapping

CLC Number: 

  • TP391
[1]CADENA C,CARLONE L,CARRILLO H,et al.Past,present,and future of simultaneous localization and mapping:Toward the robust-perception age[J].IEEE Transactions on Robotics,2016,32(6):1309-1332.
[2]LOWRY S,SÜNDERHAUF N,NEWMAN P,et al.Visualplace recognition:A survey[J].IEEE Transactions onRobo-tics,2015,32(1):1-19.
[3]SAEEDI S,PAULL L,TRENTINI M,et al.Multiple robot simultaneous localization and mapping[C]//2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.IEEE,2011:853-858.
[4]RUBLEE E,RABAUD V,KONOLIGE K,et al.ORB:An efficient alternative to SIFT or SURF[C]//2011 International Conference on Computer Vision.IEEE,2011:2564-2571.
[5]LOWE D G.Object recognition from localscale-invariant fea-tures[C]//Proceedings of the seventh IEEE International Conference on Computer Vision.IEEE,1999,2:1150-1157.
[6]GÁLVEZ-LÓPEZ D,TARDOS J D.Bags of binary words forfast place recognition in image sequences[J].IEEE Transactions on Robotics,2012,28(5):1188-1197.
[7]GARCIA-FIDALGO E,ORTIZ A.ibow-lcd:An appearance-based loop-closure detection approach using incremental bags of binary words[J].IEEE Robotics and Automation Letters,2018,3(4):3051-3057.
[8]JÉGOU H,DOUZE M,SCHMID C,et al.Aggregating local descriptors into a compact image representation[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.IEEE,2010:3304-3311.
[9]TSINTOTAS K A,BAMPIS L,GASTERATOS A.Probabilisticappearance-based place recognition through bag of tracked words[J].IEEE Robotics and Automation Letters,2019,4(2):1737-1744.
[10]WANG H,WANG C,XIE L.Online Visual Place Recognition via Saliency Re-identification[J].arXiv:2007.14549,2020.
[11]BOGOSLAVSKYI I,STACHNISS C.Fast range image-basedsegmentation of sparse 3D laser scans for online operation[C]//2016 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS).IEEE,2016:163-169.
[12]LI Y,SNAVELY N,HUTTENLOCHER D,et al.Worldwidepose estimation using 3d point clouds[C]//European Confe-rence on Computer Vision.Berlin:Springer,2012:15-29.
[13]CAMPOS C,ELVIRA R,RODRÍGUEZ J J G,et al.ORB-SLAM3:An accurate open-source library for visual,visual-inertial and multi-map SLAM[J].arXiv:2007.11898,2020.
[14]SCHLEGEL D,GRISETTI G.HBST:A hamming distance embedding binarysearch tree for feature-based visual place recognition[J].IEEE Robotics and Automation Letters,2018,3(4):3741-3748.
[15]ARANDJELOVIC R,GRONAT P,TORII A,et al.NetVLAD:CNN architecture for weakly supervised place recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:5297-5307.
[16]HAN F,EL BELEIDY S,WANG H,et al.Learning of holism-landmark graph embedding for place recognition in long-term autonomy[J].IEEE Robotics and Automation Letters,2018,3(4):3669-3676.
[17]KNEIP L,SCARAMUZZA D,SIEGWART R.A novel parametrization of the perspective-three-point problem for a direct computation of absolute camera position and orientation[C]//CVPR 2011.IEEE,2011:2969-2976.
[18]STUMM E,MEI C,LACROIX S,et al.Location graphs for vi-sual place recognition[C]//2015 IEEE International Conference on Robotics and Automation(ICRA).IEEE,2015:5475-5480.
[19]STUMM E S,MEI C,LACROIX S.Building location models for visual place recognition[J].The International Journal of Robo-tics Research,2016,35(4):334-356.
[20]MAFFRA F,TEIXEIRA L,CHEN Z,et al.Loop-closure detection in urban scenes for autonomous robot navigation[C]//2017 International Conference on 3D Vision(3DV).IEEE,2017:356-364.
[21]MAFFRA F,CHEN Z,CHLI M.tolerant PlaceRecognitioncombining 2D and 3D information for UAV navigation[C]//2018 IEEE International Conference on Robotics and Automation(ICRA).IEEE,2018:2542-2549.
[22]MAFFRA F,TEIXEIRA L,CHEN Z,et al.Real-time wide-baseline place recognition using depth completion[J].IEEE Robotics and Automation Letters,2019,4(2):1525-1532.
[23]CHAUDHURI S,RITCHIE D,WU J,et al.Learning generative models of 3D structures[J].Computer Graphics Forum,2020,39(2):643-666.
[24]ZHENG L,ZHU C,ZHANG J,et al.Active scene understan-ding via online semantic reconstruction[J].Computer Graphics Forum,2019,38(7):103-114.
[25]ZHOU T,BROWN M,SNAVELY N,et al.Unsupervised lear-ning of depth and ego-motion from video[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:1851-1858.
[26]SIVIC J,ZISSERMAN A.Video Google:A text retrieval ap-proach to object matching in videos[C]//IEEE International ConferenceonComputer Vision.IEEE Computer Society,2003:1470-1470.
[27]NISTER D,STEWENIUS H.Scalable recognition with a voca-bulary tree[C]//2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition(CVPR'06).IEEE,2006:2161-2168.
[28]GEIGER A,LENZ P,URTASUN R.Are we ready for autonomous driving? the kitti vision benchmark suite[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2012:3354-3361.
[29]BURRI M,NIKOLIC J,GOHL P,et al.The EuRoC micro aerial vehicle datasets[J].The International Journal of Robotics Research,2016,35(10):1157-1163.
[30]QIN H F,LIU X.Finger Vein Image Segmentation Based onSparse Auto-Encoder[J].Journal of Chongqing Technology and Business University(Natural Science Edition),2019,36(4):1-8.
[31]FANG L J,WANG K Q.A Visual SLAM Algorithm for Motion Redetection Combined with Deep Learning[J].Computer Engineering,2022,48(5):18-26.
[32]LIU X,WANG Z,QIN M X.Research Progress of Multi-Robot Collaborative SLAM Technology[J].Computer Enginering,2022,48(5):1-10.
[1] MA Weiqi, YUAN Jiabin, ZHA Keke, FAN Lili. Onboard Rock Detection Algorithm Based on Spiking Neural Network [J]. Computer Science, 2023, 50(1): 98-104.
[2] ZHANG Rujia, DAI Lu, GUO Peng, WANG Bang. Chinese Nested Named Entity Recognition Algorithm Based on Segmentation Attention andBoundary-aware [J]. Computer Science, 2023, 50(1): 213-220.
[3] YANG Wen-kun, YUAN Xiao-pei, CHEN Xiao-feng, GUO Rui. Spatial Multi-feature Segmentation of 3D Lidar Point Cloud [J]. Computer Science, 2022, 49(8): 143-149.
[4] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[5] SUN Fu-quan, CUI Zhi-qing, ZOU Peng, ZHANG Kun. Brain Tumor Segmentation Algorithm Based on Multi-scale Features [J]. Computer Science, 2022, 49(6A): 12-16.
[6] XU Ru-li, HUANG Zhang-can, XIE Qin-qin, LI Hua-feng, ZHAN Hang. Multi-threshold Segmentation for Color Image Based on Pyramid Evolution Strategy [J]. Computer Science, 2022, 49(6): 231-237.
[7] HU Fu-yuan, WAN Xin-jun, SHEN Ming-fei, XU Jiang-lang, YAO Rui, TAO Zhong-ben. Survey Progress on Image Instance Segmentation Methods of Deep Convolutional Neural Network [J]. Computer Science, 2022, 49(5): 10-24.
[8] CHEN Dan-hong, PENG Zhang-lin, WAN De-quan, YANG Shan-lin. Identification and Segmentation of User Value in Crowdsourcing Platforms:An Improved RFMModel [J]. Computer Science, 2022, 49(4): 37-42.
[9] GAO Xin-yue, TIAN Han-min. Droplet Segmentation Method Based on Improved U-Net Network [J]. Computer Science, 2022, 49(4): 227-232.
[10] XU Yue, ZHOU Hui1 School of Computer Science, Technology, Xi’an Jiaotong University, Xi’an 710049, China. Static Gesture Recognition Based on OpenCV in Simple Background [J]. Computer Science, 2022, 49(11A): 210800185-6.
[11] ZHANG Xin, SUN Jing, YANG Hong-bo, PAN Jia-hua, GUO Tao, WANG Wei-lian. Heart Sound Segmentation Algorithm Based on TK Energy Operator and Envelope Fusion [J]. Computer Science, 2022, 49(11A): 210900135-6.
[12] HUANG Yang-lin, HU Kai, GUO Jian-qiang, Peng ChengKey Laboratory of Intelligent Computing & Information Processing of Ministry of Education, Xiangtan University, Xiangtan, Hunan 411105,China. Liver CT Images Segmentation Based on Multi-scale Feature Fusion and Dual AttentionMechanism [J]. Computer Science, 2022, 49(11A): 210800162-9.
[13] QI Ying, CHAI Yan-mei. High-resolution Remote Sensing Sea Ice Image Segmentation Based on Combination of ImprovedSLIC Algorithm and Clustering Algorithm [J]. Computer Science, 2022, 49(11A): 211200100-6.
[14] JIN Yu-jie, CHU Xu, WANG Ya-sha, ZHAO Jun-feng. Variational Domain Adaptation Driven Semantic Segmentation of Urban Scenes [J]. Computer Science, 2022, 49(11): 126-133.
[15] ZHANG Fu-chang, ZHONG Guo-qiang, MAO Yu-xu. Neural Architecture Search for Light-weight Medical Image Segmentation Network [J]. Computer Science, 2022, 49(10): 183-190.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!