计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (8): 183-191.doi: 10.11896/jsjkx.230500094

• 计算机图形学&多媒体 • 上一篇    下一篇

嵌入注意力机制的并行多尺度点云上采样方法

肖霄, 柏正尧, 李泽锴, 刘旭珩, 杜佳锦   

  1. 云南大学信息学院 昆明 650500
  • 收稿日期:2023-05-15 修回日期:2023-10-11 出版日期:2024-08-15 发布日期:2024-08-13
  • 通讯作者: 柏正尧(baizhy@ynu.edu.cn)
  • 作者简介:(xenon@mail.ynu.edu.cn)
  • 基金资助:
    云南省重大科技专项计划(202002AD080001)

Parallel Multi-scale with Attention Mechanism for Point Cloud Upsampling

XIAO Xiao, BAI Zhengyao, LI Zekai, LIU Xuheng, DU Jiajin   

  1. School of Information Science and Engineering,Yunnan University,Kunming 650500,China
  • Received:2023-05-15 Revised:2023-10-11 Online:2024-08-15 Published:2024-08-13
  • About author:XIAO Xiao,born in 1999,postgraduate.His main research interests include three-dimensional reconstruction and 3D point cloud upsampling.
    BAI Zhengyao,born in 1967,Ph.D,professor,master supervisor.His main research interests include signal proces-sing,image processing,pattern recognition and machine learning,etc.
  • Supported by:
    Yunnan Provincial Major Science and Technology Special Plan(202002AD080001).

PDF (PC)

365

摘要： 目前,基于深度学习的点云上采样方法缺失对局部区域特征关联性的关注和对全局特征的多尺度提取,导致输出的密集点云存在异常值过多、细粒度不高等问题。为解决上述问题,提出了嵌入注意力机制的并行多尺度点云上采样网络(Parallel Multi-scale with Attention mechanism for Point cloud Upsampling),网络由特征提取器、特征拓展器、坐标细化器和坐标重建器4个模块级联组成。首先给定一个N×3的稀疏点云作为输入,为了获得点云的全局和局部特征信息,设计了一个嵌入注意力机制的并行多尺度特征提取模块(PMA)用于将三维空间的点云映射到高维特征空间。其次使用边缘卷积特征拓展器拓展点云特征维度,得到高维点云特征,以更好地保留点云特征的边缘信息,将高维点云特征通过坐标重建器转换回三维空间中。最后使用坐标细化器精细调整输出点云细节。在合成数据集PU1K上的对比实验结果表明,PMA-PU生成的密集点云在倒角距离(CD)、豪斯多夫距离(HD)和点面距离(P2F)上都有显著提升,分别比性能次优的网络模型优化了7.863%,21.631%,14.686%。可视化结果证明了PMA-PU具有性能更好的特征提取器,能够生成细粒度更高、形状更接近真实值的密集点云。

关键词: 3D点云, 深度学习, 点云上采样, 并行多尺度特征提取, 注意力机制

Abstract: The current deep learning-based point cloud upsampling method lacks the attention to a local area feature correlation and multi-scale extraction of global features,resulting in the dense output point cloud with too many outliers and low fine-grained granularity.To solve the above problem,a parallel multi-scale with attention mechanism for point cloud upsampling(PMA-PU) network is proposed,which consists of a feature extractor,a feature expander,a coordinate refiner and a coordinate reconstructor.Firstly,giving an N×3 sparse point cloud as input,a parallel multi-scale feature extraction module(PMA) with an embedded attention mechanism is designed to map the point cloud in 3D space to the high-dimensional feature space to obtain the global and local feature information of the point cloud.Secondly,the high-dimensional point cloud features are obtained after expanding the dimensionality of the point cloud features using the edge convolution feature expander to better preserve the edge information of the point cloud features,and the high-dimensional point cloud features are converted back to the 3D space by the coordinate reconstructors.Finally,the output point cloud details are fine-tuned by using the coordinate refiners.The results of the PMA-PU comparison experiments on the synthetic dataset PU1K show that the generated dense point cloud has significant improvement in the three evaluation metrics,Chamfer Distance(CD),Hausdorff Distance(HD),and P2F(point-to-surface),which are significantly better than the second highest performance.The network models with the second highest performance are optimized by 7.863%,21.631%,and 14.686%,respectively.The visualization results demonstrate that PMA-PU has a better performce feature extractor,which can generate dense point clouds with higher fine granularity and a shape closer to the true value.

Key words: 3D point cloud, Deep learning, Point cloud upsampling, Parallel multi-scale feature extraction, Attention mechanism

中图分类号: 

  • TP391
[1]XU S K,ZHANG L J,SHI L,et al.Few-shot 3D point cloud object detection guided by Intention-attention[J].Computer Engineering,doi:10.19678/j.issn.1000-3428.0068727.
[2]YANG W K,YUAN X P,CHEN X F.Spatial Multi-featureSegmentation of 3D Lidar Point Cloud[J].Computer Science,2021,49(8):143-149.
[3]YU L Q,LI X Z,FU C W,et al.PU-Net:Point cloud upsampling network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:2790-2799.
[4]WANG Y F,WU S H,HUANG H,et al.Patch-based progressive 3d point set upsampling[C]//Proceedings of IEEE Confe-rence on Computer Vision andPattern Recognition.2019:5958-5967.
[5]QIAN G C,ABUALSHOUR A,LI G H,et al.PU-GCN:point cloud upsampling using graph convolutional networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:11683-11692.
[6]HAN B,ZHANG X Y,REN S.PU-GACNet:Graph Attentionconvolution Network for Point Cloud Upsampling[J].Image and Vision Computing,2022,118:104371.
[7]ALEXA M,BEHR J,COHEN-OR D,et al.Computing and rendering point set surfaces[J]IEEE Transactions on Visualization and Computer Graphics,2003,9(1):3-15.
[8]FERSTL D,REINBACHER C,RANFTL R,et al.Image guided depth upsampling using anisotropic total generalized variation[C]//Proceedings of IEEE International Conference on Computer Vision.2013:993-1000.
[9]SCHUON S,THEOBALT C,DAVIS J,et al.High-quality scan-ning using time-of-flight depth super resolution[C]//2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.2008:1-7.
[10]QI C R,SU H,MO K C,et al.Pointnet:Deep learning on point sets for 3d classification and segmentation[C]//Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition.2017:652-660.
[11]QI C R,LI Y,HAO S,et al.PointNet++:Deep Hierarchical Feature Learning on Point sets in a metric space[C]//Procee-dings of the 31st International Conference on Neural Information Processing Systems.2017:5105-5114.
[12]LI R H,LI X Z,FU C W,et al.PU-GAN:a point cloud upsampling adversarial network[C]//Proceedings of IEEE/CVF International Conference on Computer Vision.2019:7203-7212.
[13]LI R H,LI X Z,HENG P A,et al.Point cloud upsampling viadisentangled refinement[C]//Proceedings of IEEE/CVF Conference on Computer Vision andPattern Recognition.2021:344-353.
[14]LIU H,YUAN H,HOU J,et al.PUFA-GAN:A Frequency-Aware Generative Adversarial Network for 3D point Cloud Upsampling[J].IEEE Transactions on Image Processing,2022,31:7389-7402.
[15]LIU H,YUAN H,HAMZAOUI R,et al.PU-Refiner:A Geo-metry Refiner with Adversarial Learning for Point Cloud Upsampling[C]//IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP 2022).2022:2270-2274.
[16]LUO L Q,TANG L L,ZHOU W Y,et al.PU-EVA:an edge-vector based approximation solution for flexible-scale point cloud upsampling[C]//Proceedings of IEEE/CVF International Conference on Computer Vision.2021:16208-16217.
[17]ZHOU J,CUI G Q,HU S D,et al.Graph neural networks:a review of methods and applications[J].AI Open,2020,1:57-81.
[18]XU Y F,FAN T Q,XU M Y,et al.SpiderCNN:deep learning on point sets with parameterized convolutional filters[C]//Proceedings of European Conference on Computer Vision(ECCV).2018:87-102.
[19]MAO J,WANG X G,LI H S.Interpolated convolutional networksfor 3d point cloud understanding[C]//Proceedings of IEEE/CVF International Conference onComputer Vision.2019:1578-1587.
[20]WANG Y,SUN Y B,LIU Z W,et al.Dynamic graph cnn for learning on point clouds[J].ACM Transactions on Graphics(ToG),2019,38(5):1-12.
[21]LI G H,MÜLLER M,THABET A,et al.DeepGCNs:Can GCNs go as deep as CNNs?[C]//Proceedings of IEEE/CVF International Conference on Computer Vision.2019:9267-9276.
[22]KAZI A,SHEKARFOROUSH S,KRISHNA S A,et al.InceptionGCN:Receptive field aware graph convolutional network for disease prediction[C]//Processingsof 26th International Confe-rence on Medical Imaging.2019:73-85.
[23]WANG L,HUANG Y C,HOU Y L,et al.Graph attention convolution for point cloud semantic segmentation[C]//Procee-dings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:10296-10305.
[24]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[C]//Proceedings of the 31st International Confe-rence on Neural Information Processing Systems.2017:6000-6010.
[25]ZHONG F,BAI Z Y.3D point cloud super resolution with dynamic residual graph convolution networks[J].Journal of Zhejiang University(Engineering Science),2022,56(11):2251-2259.
[26]LIU X H,LIU X C,LIU Y S,et al.SPU-Net:Self-Supervised Point Cloud Upsampling by Coarse-to-Fine Reconstruction With Self-Projection Optimization[J].IEEE Transactions on Image Processing,2022,31:4213-4226.
[27]CHANG A X,FUNKHOUSER T,GUIBAS L,et al.ShapeNet:An Information-Rich 3D ModelRepository[J].arXiv:1512.03012,2015.
[28]WIEMANN T,MITSCHKE I,MOCK A,et al.Surface Reconstruction from Arbitrarily Large Point Clouds[C]//2018 Second IEEE International Conference on Robotic Computing(IRC).2018:278-281.
[29]HOPPE H.Poisson surface reconstruction and its applications[C]//Solid and Physical Modeling.2008.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发