计算机科学

计算机科学 ›› 2023, Vol. 50 ›› Issue (6A): 220400214-7.doi: 10.11896/jsjkx.220400214

• 图像处理&多媒体技术 • 上一篇    下一篇

基于激光雷达点云的3D目标检测方法综述

秦静1, 王伟滨2, 邹启杰2, 汪祖民2, 季长清3   

  1. 1 大连大学软件工程学院 辽宁 大连116622;
    2 大连大学信息工程学院 辽宁 大连 116622;
    3 大连大学物理科学与技术学院 辽宁 大连 116622
  • 出版日期:2023-06-10 发布日期:2023-06-12
  • 通讯作者: 汪祖民(wangzumin@dlu.edu.cn)
  • 作者简介:(qinjing@dlu.edu.cn)
  • 基金资助:
    国家自然科学基金青年科学基金项目(62002038)

Review of 3D Target Detection Methods Based on LiDAR Point Clouds

QIN Jing1, WANG Weibin2, ZOU Qijie2, WANG Zumin2, JI Changqing3   

  1. 1 College of Software Engineering,Dalian University,Dalian,Liaoning 116622,China;
    2 College of Information Engineering,Dalian University,Dalian,Liaoning 116622,China;
    3 College of Physical Science and Technology,Dalian University,Dalian,Liaoning 116622,China
  • Online:2023-06-10 Published:2023-06-12
  • About author:QIN Jing,born in 1981,Ph.D,associate professor,is a member of China Computer Federation.Her main research interests include signal processing and big data analysis. WANG Zumin,born in 1975,Ph.D,professor,is a member of China Computer Federation.His main research interests include smart cities and internet of things.
  • Supported by:
    National Science Foundation of China(62002038).

PDF (PC)

412

摘要： 近年来,利用激光雷达点云进行3D目标检测是计算机视觉领域的一个研究热点,并在自动驾驶领域备受关注。3D相比于2D而言,结合了深度信息,更能体现出现实世界的特征,以有效解决如路径规划、运动预测、目标检测等方面的实际问题。介绍了3D目标检测的发展背景,概述了基于激光雷达点云数据的3D目标检测框架的流程,比较了几种常见的包含点云信息的数据集,并对主要研究方法进行分类。结合了自动驾驶的应用场景,对不同方法的性能和局限性进行了分析和比较。最后,总结了现阶段的技术难点,并展望了该领域未来的发展前景。

关键词: 目标检测, 点云, 计算机视觉, 激光雷达, 多模态融合

Abstract: In recent years,3D target detection using LiDAR point cloud is a research hotspot in the field of computer vision and has attracted much attention in the field of autonomous driving.Compared with 2D,3D combines depth information to better reflect the characteristics of the real world,to effectively solve practical problems such as path planning,motion prediction,target detection,and other aspects.This paper introduces the development background of 3D target detection,summarizes the flow of 3D target detection framework based on LiDAR point cloud data,compares several common data sets containing point cloud information,and classifies the main research methods.The performance and limitations of different methods are analyzed and compared.Finally,the current technical difficulties are summarized and the future development prospects of this field are forecasted.

Key words: Target detection, Point cloud, Computer vision, LiDAR, Multimodal fusion

中图分类号: 

  • TP391.41
[1]LIU W,ANGUELOV D,ERHAN D,et al.Ssd:Single shotmultibox detector[C]//European Conference on Computer Vision.2016:21-37.
[2]REDMON J,DIVVALA S,GIRSHICK R,et al.You only look once:Unified,real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:779-788.
[3]REN S,HE K,GIRSHICK R,et al.Faster r-cnn:Towards real-time object detection with region proposal networks[J].Advances in Neural Information Processing systems,2015,28.
[4]MASI I,WU Y,HASSNER T,et al.Deep face recognition:A survey[C]//2018 31st SIBGRAPI Conference on Graphics,Patterns and mages(SIBGRAPI).IEEE,2018:471-478.
[5]HUANG R,GU J,SUN X,et al.A rapid recognition method for electronic components based on the improved YOLO-V3 network[J].Electronics,2019,8(8):825.
[6]YURTSEVER E,LAMBERT J,CARBALLO A,et al.A survey of autonomous driving:Common practices and emerging techno-logies[J].IEEE access,2020,8:58443-58469.
[7]LIANG M,YANG B,ZENG W,et al.Pnpnet:End-to-end perception and prediction with tracking in the loop[C]//Procee-dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:11553-11562.
[8]XIAO Y Q,YANG H M.Research on Application of Object Detection Algorithm in Traffic Scene[J].Computer Engineering and Applications,2021,57(6):30-41.
[9]BRESSON G,ALSAYED Z,YU L,et al.Simultaneous localization and mapping:A survey of current trends in autonomousdriving[J].IEEE Transactions on Intelligent Vehicles,2017,2(3):194-220.
[10]BOJARSKI M,DEL TESTA D,DWORAKOWSKI D,et al.End to end learning for self-driving cars[J].arXiv:1604.07316,2016.
[11]HECHT J.Lidar for self-driving cars[J].Optics and Photonics News,2018,29(1):26-33.
[12]PARK Y,YUN S,WON C S,et al.Calibration between color camera and 3D LIDAR instruments with a polygonal planar board[J].Sensors,2014,14(3):5333-5353.
[13]SOMMER H,KHANNA R,GILITSCHENSKI I,et al.A low-cost system for high-rate,high-accuracy temporal calibration for LIDARs and cameras[C]//2017 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS).2017:2219-2226.
[14]CUI Y,CHEN R,CHU W,et al.Deep learning for image and point cloud fusion in autonomous driving:A review[J].IEEE Transactions on Intelligent Transportation Systems,2021,23(2):722-739.
[15]CHAZETTE P,TOTEMS J,HESPEL L,et al.Principle and physics of the LiDAR measurement[M]//Optical Remote Sen-sing of Land Surface.Elsevier,2016:201-247.
[16]HE H,WANG H,SUN L.Research on 3D point-cloud registration technology based on Kinect V2 sensor[C]//2018 Chinese Control And Decision Conference(CCDC).IEEE,2018:1264-1268.
[17]XU Y,JOHN V,MITA S,et al.3D point cloud map based vehicle localization using stereo camera[C]//2017 IEEE Intelligent Vehicles Symposium(IV).IEEE,2017:487-492.
[18]LI G Y,LI M L,WANG L,et al.AReview of Preprocessing of Laser-scanned Point Clouds Data[J].Bulletin of Surveying and Mapping,2015(11):1-3.
[19]XU Z,ZHANG Z,ZHONG R,et al.Content-sensitive multilevel point cluster construction for ALS point cloud classification[J].Remote Sensing,2019,11(3):342.
[20]RUSU R B,COUSINS S.3d is here:Point cloud library(pcl)[C]//2011 IEEE International Conference on Robotics and Automation.2011:1-4.
[21]GEIGER A,LENZ P,URTASUN R.Are we ready for autonomous driving? the kitti vision benchmarksuite[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition.2012:3354-3361.
[22]CAESAR H,BANKITI V,LANG A H,et al.nuscenes:A multimodal dataset for autonomous driving[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:11621-11631.
[23]SUN P,KRETZSCHMAR H,DOTIWALLA X,et al.Scalability in perception for autonomous driving:Waymo open dataset[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:2446-2454.
[24]QI C R,SU H,MO K,et al.Pointnet:Deep learning on point sets for 3d classification and segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:652-660.
[25]QI C R,YI L,SU H,et al.Pointnet++:Deep hierarchical feature learning on point sets in a metric space[C]//Conference and Workshop on Neural Information Processing Systems.2017.
[26]SHI S,WANG X,LI H.Pointrcnn:3d object proposal generation and detection from point cloud[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:770-779.
[27]YANG Z,SUN Y,LIU S,et al.3dssd:Point-based 3d single stage object detector[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:11040-11048.
[28]ZHOU Y,TUZEL O.Voxelnet:End-to-end learning for point cloud based 3d object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:4490-4499.
[29]YAN Y,MAO Y,LI B.Second:Sparsely embedded convolutionaldetection[J].Sensors,2018,18(10):3337.
[30]LANG A H,VORA S,CAESAR H,et al.Pointpillars:Fast encoders for object detection from point clouds[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:12697-12705.
[31]YE M,XU S,CAO T.Hvnet:Hybrid voxel network for lidar based 3d object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:1631-1640.
[32]SHI S,WANG Z,SHI J,et al.From points to parts:3d object detection from point cloud with part-aware and part-aggregation network[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,43(8):2647-2664.
[33]HE C,ZENG H,HUANG J,et al.Structure aware single-stage 3d objectdetection from point cloud[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:11873-11882.
[34]GUSTAFSSON F K,DANELLJAN M,SCHON T B.Accurate 3D object detection using energy-based models[C]//Proceedings ofthe IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:2855-2864.
[35]ZHENG W,TANG W,CHEN S,et al.CIA-SSD:Confident IoU-aware single-stage object detector from point cloud[J].arXiv:2012.03015,2020.
[36]ZHENG W,TANG W,JIANG L,et al.SE-SSD:Self-ensembling single-stage object detector from point cloud[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:14494-14503.
[37]LI Z,YAO Y,QUAN Z,et al.Sienet:spatial information enhancement network for 3d object detection from point cloud[J].arXiv:2103.15396,2021.
[38]CHEN Q,SUN L,WANG Z,et al.Object as hotspots:An anchor-free 3d object detection approach via firing of hotspots[C]//European Conference on Computer Vision.2020:68-84.
[39]QIAN R,LAI X,LI X.Boundary-aware 3d object detection from point clouds[J].arXiv:2104.10330,2021.
[40]LI J,DAI H,SHAO L,et al.From voxel to point:Iou-guided 3d object detection for point cloud with voxel-to-point decoder[C]//Proceedings of the 29th ACM International Conference on Multimedia.2021:4622-4631.
[41]DENG J,ZHOU W,ZHANG Y,et al.From Multi-View to Hol-low-3D:Hallucinated Hollow-3D R-CNN for 3D Object Detection[J].IEEE Transactions on Circuits and Systems for Video Technology,2021,31(12):4722-4734.
[42]XIE L,XU G,CAI D,et al.X-view:Non-egocentric Multi-View 3D Object Detector[J].arXiv:2103.13001,2021.
[43]CHEN X,MA H,WAN J,et al.Multi-view 3dobject detection network for autonomous driving[C]//Proceedings of the IEEE conference on Computer Vision and Pattern Recognition.2017:1907-1915.
[44]KU J,MOZIFIAN M,LEE J,et al.Joint 3d proposal generation and object detection from view aggregation[C]//2018 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS).2018:1-8.
[45]YOO J H,KIM Y,KIM J,et al.3d-cvf:Generating joint camera and lidar features using cross-view spatial feature fusion for 3d object detection[C]//European Conference on Computer Vision.2020:720-736.
[46]PANG S,MORRIS D,RADHA H.CLOCs:Camera-LiDAR object candidates fusion for 3D object detection[C]//2020 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS).2020:10386-10393.
[47]KU J,PON A D,WALSH S,et al.Improving 3d object detection for pedestrians with virtual multi-view synthesis orientation estimation[C]//2019 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS).2019:3459-3466.
[48]VORA S,LANG A H,HELOU B,et al.Pointpainting:Sequential fusion for 3d object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:4604-4612.
[49]FEI J,CHEN W,HEIDENREICH P,et al.SemanticVoxels:Sequential fusion for 3D pedestrian detection using LiDAR point cloud and semantic segmentation[C]//2020 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems(MFI).2020:185-190.
[50]QI C R,LIU W,WU C,et al.Frustum pointnets for 3d object detection from rgb-d data[C]//Proceedings of the IEEE Confe-rence on Computer Vision and Pattern Recognition.2018:918-927.
[51]WANG Z,JIA K.Frustum convnet:Slidingfrustums to aggregate local point-wise features for amodal 3d object detection[C]//2019 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS).2019:1742-1749.
[52]PAIGWAR A,SIERRA-GONZALEZ D,ERKENT Ö,et al.Frustum-pointpillars:A multi-stage approach for 3d object detection using rgb camera and lidar[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:2926-2933.
[53]SHI S,GUO C,JIANG L,et al.Pv-rcnn:Point-voxel feature set abstraction for 3d object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:10529-10538.
[54]LI J,SUN Y,LUO S,et al.P2V-RCNN:Point to Voxel Feature Learning for 3D Object Detection from Point Clouds[J].IEEE Access,2021,9:98249-98260.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发