基于动态图卷积和空间金字塔池化的点云深度学习网络

doi:10.11896/jsjkx.190700180

计算机科学 ›› 2020, Vol. 47 ›› Issue (7): 192-198.doi: 10.11896/jsjkx.190700180

基于动态图卷积和空间金字塔池化的点云深度学习网络

朱威^1,2, 绳荣金¹, 汤如¹, 何德峰^1,2

1 浙江工业大学信息工程学院杭州310023
2 浙江省嵌入式系统联合重点实验室杭州310023

收稿日期:2019-07-26 出版日期:2020-07-15 发布日期:2020-07-16
通讯作者: 朱威(weizhu@zjut.edu.cn)
基金资助:
浙江省自然科学基金(LY17F010013);国家自然科学基金(61401398)

Point Cloud Deep Learning Network Based on Dynamic Graph Convolution and Spatial Pyramid Pooling

ZHU Wei^1,2, SHENG Rong-jin¹, TANG Ru¹, HE De-feng^1,2

1 College of Information Engineering,Zhejiang University of Technology,Hangzhou 310023,China
2 United Key Laboratory of Embedded System of Zhejiang Province,Hangzhou 310023,China

Received:2019-07-26 Online:2020-07-15 Published:2020-07-16
About author:ZHU Wei,born in 1982,Ph.D,associate professor.His main research interests include video processing,machine learning and intelligent robot.
Supported by:
This work was supported by the Natural Science Foundation of Zhejiang Province (LY17F010013) and National Natural Science Foundation of China (61401398)

摘要/Abstract

摘要： 点云数据的分类和语义分割在自动驾驶、智能机器人、全息投影等领域中有着重要应用。传统手工提取点云特征的方式,以及将三维点云数据转化为多视图、体素网格等数据形式后再进行特征学习的方式,都存在处理环节多、三维特征损失大等问题,分类和分割的精度较低。目前可以直接处理点云数据的深度神经网络PointNet忽略了点云的局部细粒度特征,对复杂点云场景的处理能力较弱。针对上述问题,提出了一种基于动态图卷积和空间金字塔池化的点云深度学习网络。该网络在PointNet的基础上使用动态图卷积模块来替换PointNet中的特征学习模块,增强了网络对局部拓扑结构信息的学习能力;同时设计了一种基于点的空间金字塔池化结构来捕获多尺度局部特征,该方式比PointNet++的多尺度采样点云、重复分组进行多尺度局部特征学习的方法更加简洁高效。实验结果表明,在点云分类和语义分割任务的3个基准数据集上,所提网络相较于现有网络具有更高的分类和分割精度。

关键词: PointNet, 点云, 动态图卷积, 局部特征, 空间金字塔池化

Abstract: The classification and semantic segmentation of point cloud data have important applications in automatic driving,intelligent robot and holographic projection.While using the traditional method of manually extracting point cloud features or the feature learning method of firstly transforming three-dimensional point cloud data into data forms of multi-view and volumetric grid,there exist problems such as many processing links and great loss of three-dimensional features,resulting in low accuracy of classification and segmentation.The existing deep neural network PointNet,which can directly process point cloud data,ignoresthe local fine-grained features of point cloud and is weak in processing complex point cloud scenarios.To solve the above problems,this paper proposes a point cloud deep learning network based on dynamic graph convolution and spatial pyramid pooling.On the basis of PointNet,the dynamic graph convolution module GraphConv is used to replace the feature learning module in PointNet,which enhances the network’s ability to learn local topological structure information.At the same time,a point-based spatial py-ramid pooling structure PSPP is designed to capture multi-scale local features.Compared with the multi-scale sampling point cloud of PointNet++ and the repeated grouping method for multi-scale local features learning,it is simpler and more efficient.Experimental results show that,on the three benchmark data sets of point cloud classification and semantic segmentation task,the proposed network has higher classification and segmentation accuracy than the existing network.

Key words: Dynamic graph convolution, Local features, Point cloud, PointNet, Spatial pyramid pooling

中图分类号:

TP391

朱威, 绳荣金, 汤如, 何德峰. 基于动态图卷积和空间金字塔池化的点云深度学习网络[J]. 计算机科学, 2020, 47(7): 192-198. https://doi.org/10.11896/jsjkx.190700180

ZHU Wei, SHENG Rong-jin, TANG Ru, HE De-feng. Point Cloud Deep Learning Network Based on Dynamic Graph Convolution and Spatial Pyramid Pooling[J]. Computer Science, 2020, 47(7): 192-198. https://doi.org/10.11896/jsjkx.190700180

参考文献

[1]LIU J,WU Z K,ZHOU M Q.Overview of point cloud modelsegmentation and application technology[J].Computer Science,2011,38(4):21-24.
[2]QI C R,SU H,KAICHUN M,et al.PointNet:deep learning on point sets for 3D classification and segmentation[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition.Honolulu:IEEE Computer Society Press,2017:77-85.
[3]SU H,MAJI S,KALOGERAKIS E,et al.Multi-view convolutional neural networks for 3D shape recognition [C]//2015 IEEE International Conference on Computer Vision.New York:IEEE Press,2015:945-953.
[4]KLOKOV R,LEMPITSKY V.Escape from cells:Deep kd-networks for the recognition of 3d point cloud models[C]//Proceedings of the IEEE International Conference on Computer Vision.Honolulu:IEEE Computer Society Press,2017:863-872.
[5]XU Y,FAN T,XU M,et al.Spidercnn:Deep learning on point sets with parameterized convolutional filters[C]//Proceedings of the European Conference on Computer Vision (ECCV).Munich:IEEE Press,2018:87-102.
[6]CHEN C,LUCA Z F,ANTONIOS T.GAPNet:Graph Attention based Point Neural Network for Exploiting Local Feature of Point Cloud[J].arXiv:1905.08705.
[7]BAI J,SI Q L,QIN F Y.LightPointNet,a lightweight real-time point cloud classification network[J].Journal of Computer-Aided Design and Graphics,2019,31(4):612-621.
[8]QI C R,SU H,NIEβNER M,et al.Volumetric and multi-view cnns for object classification on 3d data[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE Computer Society Press,2016:5648-5656.
[9]MATURANA D,SCHERER S.Voxnet:A 3d convolutionalneural network for real-time object recognition[C]//2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).New York:IEEE Press,2015:922-928.
[10]WU Z,SONG S,KHOSLA A,et al.3d shapenets:A deep representation for volumetric shapes[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Boston:IEEE Computer Society Press,2015:1912-1920.
[11]JADERBERG M,SIMONYAN K,ZISSERMAN A.Spatialtransformer networks[C]//The 24th Annual Conference on Neural Information Processing Systems.Cambridge:MIT Press,2015:2017-2025.
[12]QI C R,YI L,SU H,et al.PointNet++:Deep hierarchical feature learning on point sets in a metric space[C]//The 24th Annual Conference on Neural Information Processing Systems.Cambridge:MIT Press,2017:5105-5114.
[13]DEFFERRARD M,BRESSON X,VANDERGHEYNST P.Convolutional neural networks on graphs with fast localized spectral filtering[C]//Advances in Neural Information Processing Systems.New York:IEEE Press,2016:3844-3852.
[14]SIMONOVSKY M,KOMODAKIS N.Dynamic edge-conditioned filters in convolutional neural networks on graphs[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Honolulu:IEEE Computer Society Press,2017:3693-3702.
[15]WANG Y,SUN Y,LIU Z,et al.Dynamic Graph CNN forLearning on Point Clouds[J].arXiv:1801.07829.
[16]HE K,ZHANG X,REN S,et al.Spatial pyramid pooling in deep convolutional networks for visual recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(9):1904-1916.
[17]YI L,GUIBAS L,KIM V G,et al.A scalable active framework for region annotation in 3D shape collections[J].ACM Transactions on Graphics,2016,35(6):1-12.
[18]ARMENI I,SENER O,ZAMIR A R,et al.3d semantic parsing of large-scale indoor spaces[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE Computer Society Press,2016:1534-1543.

相关文章 15

[1]	李宗民, 张玉鹏, 刘玉杰, 李华. 基于可变形图卷积的点云表征学习 Deformable Graph Convolutional Networks Based Point Cloud Representation Learning 计算机科学, 2022, 49(8): 273-278. https://doi.org/10.11896/jsjkx.210900023
[2]	杨文坤, 原晓佩, 陈小锋, 郭睿. 三维激光雷达点云空间多特征分割 Spatial Multi-feature Segmentation of 3D Lidar Point Cloud 计算机科学, 2022, 49(8): 143-149. https://doi.org/10.11896/jsjkx.210300275
[3]	孟月波, 穆思蓉, 刘光辉, 徐胜军, 韩九强. 基于向量注意力机制GoogLeNet-GMP的行人重识别方法 Person Re-identification Method Based on GoogLeNet-GMP Based on Vector Attention Mechanism 计算机科学, 2022, 49(7): 142-147. https://doi.org/10.11896/jsjkx.210600198
[4]	封雷, 朱登明, 李兆歆, 王兆其. 一种基于遮罩的稀疏点云滤波算法 Sparse Point Cloud Filtering Algorithm Based on Mask 计算机科学, 2022, 49(5): 25-32. https://doi.org/10.11896/jsjkx.210600129
[5]	徐涛, 陈奕仁, 吕宗磊. 基于改进YOLOv3的机坪工作人员反光背心检测研究 Study on Reflective Vest Detection for Apron Workers Based on Improved YOLOv3 Algorithm 计算机科学, 2022, 49(4): 239-246. https://doi.org/10.11896/jsjkx.210200119
[6]	赵新灿, 常寒星, 金仁标. 3D点云形状补全GAN 3D Point Cloud Shape Completion GAN 计算机科学, 2021, 48(4): 192-196. https://doi.org/10.11896/jsjkx.200100048
[7]	姚楠, 张征. 基于三维图像的疤痕面积计算 Scar Area Calculation Based on 3D Image 计算机科学, 2021, 48(11A): 308-313. https://doi.org/10.11896/jsjkx.201100044
[8]	唐一星, 刘学亮, 胡社教. 多方向分区网络结构的行人再识别 Multi-orientation Partitioned Network for Person Re-identification 计算机科学, 2021, 48(10): 204-211. https://doi.org/10.11896/jsjkx.210300128
[9]	许华杰, 杨洋, 李桂兰. 基于注意力机制和深度卷积神经网络的材质识别方法 Material Recognition Method Based on Attention Mechanism and Deep Convolutional Neural Network 计算机科学, 2021, 48(10): 220-225. https://doi.org/10.11896/jsjkx.200800073
[10]	曾俊飞,杨海清,吴浩. 面向三维重建的自适应列文伯格-马夸尔特点云配准方法 Adaptive Levenberg-Marquardt Cloud Registration Method for 3D Reconstruction 计算机科学, 2020, 47(3): 137-142. https://doi.org/10.11896/jsjkx.190200261
[11]	史文凯, 张昭晨, 喻孟娟, 吴瑞, 聂建辉. 基于特征检测与深度特征描述的点云粗对齐算法 Point Cloud Coarse Alignment Algorithm Based on Feature Detection and Depth FeatureDescription 计算机科学, 2020, 47(12): 252-257. https://doi.org/10.11896/jsjkx.191000069
[12]	李健, 杨祥如, 何斌. 基于深度学习的几何特征匹配方法 Geometric Features Matching with Deep Learning 计算机科学, 2019, 46(7): 274-279. https://doi.org/10.11896/j.issn.1002-137X.2019.07.042
[13]	吴飞, 赵新灿, 展鹏磊, 关凌. 自适应邻域选择的FPFH特征提取算法 FPFH Feature Extraction Algorithm Based on Adaptive Neighborhood Selection 计算机科学, 2019, 46(2): 266-270. https://doi.org/10.11896/j.issn.1002-137X.2019.02.041
[14]	刘振宇, 关彤. 基于RGB-D图像的头部姿态检测 Head Posture Detection Based on RGB-D Image 计算机科学, 2019, 46(11A): 334-340.
[15]	王岩, 罗倩, 邓辉. 基于变分贝叶斯的轴承故障诊断方法 Bearing Fault Diagnosis Method Based on Variational Bayes 计算机科学, 2019, 46(11): 323-327. https://doi.org/10.11896/jsjkx.180901719

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

基于动态图卷积和空间金字塔池化的点云深度学习网络

Point Cloud Deep Learning Network Based on Dynamic Graph Convolution and Spatial Pyramid Pooling

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

Metrics

本文评价

推荐阅读 0