Computer Science

Computer Science ›› 2024, Vol. 51 ›› Issue (8): 183-191.doi: 10.11896/jsjkx.230500094

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Parallel Multi-scale with Attention Mechanism for Point Cloud Upsampling

XIAO Xiao, BAI Zhengyao, LI Zekai, LIU Xuheng, DU Jiajin   

  1. School of Information Science and Engineering,Yunnan University,Kunming 650500,China
  • Received:2023-05-15 Revised:2023-10-11 Online:2024-08-15 Published:2024-08-13
  • About author:XIAO Xiao,born in 1999,postgraduate.His main research interests include three-dimensional reconstruction and 3D point cloud upsampling.
    BAI Zhengyao,born in 1967,Ph.D,professor,master supervisor.His main research interests include signal proces-sing,image processing,pattern recognition and machine learning,etc.
  • Supported by:
    Yunnan Provincial Major Science and Technology Special Plan(202002AD080001).

PDF (PC)

362

Abstract: The current deep learning-based point cloud upsampling method lacks the attention to a local area feature correlation and multi-scale extraction of global features,resulting in the dense output point cloud with too many outliers and low fine-grained granularity.To solve the above problem,a parallel multi-scale with attention mechanism for point cloud upsampling(PMA-PU) network is proposed,which consists of a feature extractor,a feature expander,a coordinate refiner and a coordinate reconstructor.Firstly,giving an N×3 sparse point cloud as input,a parallel multi-scale feature extraction module(PMA) with an embedded attention mechanism is designed to map the point cloud in 3D space to the high-dimensional feature space to obtain the global and local feature information of the point cloud.Secondly,the high-dimensional point cloud features are obtained after expanding the dimensionality of the point cloud features using the edge convolution feature expander to better preserve the edge information of the point cloud features,and the high-dimensional point cloud features are converted back to the 3D space by the coordinate reconstructors.Finally,the output point cloud details are fine-tuned by using the coordinate refiners.The results of the PMA-PU comparison experiments on the synthetic dataset PU1K show that the generated dense point cloud has significant improvement in the three evaluation metrics,Chamfer Distance(CD),Hausdorff Distance(HD),and P2F(point-to-surface),which are significantly better than the second highest performance.The network models with the second highest performance are optimized by 7.863%,21.631%,and 14.686%,respectively.The visualization results demonstrate that PMA-PU has a better performce feature extractor,which can generate dense point clouds with higher fine granularity and a shape closer to the true value.

Key words: 3D point cloud, Deep learning, Point cloud upsampling, Parallel multi-scale feature extraction, Attention mechanism

CLC Number: 

  • TP391
[1]XU S K,ZHANG L J,SHI L,et al.Few-shot 3D point cloud object detection guided by Intention-attention[J].Computer Engineering,doi:10.19678/j.issn.1000-3428.0068727.
[2]YANG W K,YUAN X P,CHEN X F.Spatial Multi-featureSegmentation of 3D Lidar Point Cloud[J].Computer Science,2021,49(8):143-149.
[3]YU L Q,LI X Z,FU C W,et al.PU-Net:Point cloud upsampling network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:2790-2799.
[4]WANG Y F,WU S H,HUANG H,et al.Patch-based progressive 3d point set upsampling[C]//Proceedings of IEEE Confe-rence on Computer Vision andPattern Recognition.2019:5958-5967.
[5]QIAN G C,ABUALSHOUR A,LI G H,et al.PU-GCN:point cloud upsampling using graph convolutional networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:11683-11692.
[6]HAN B,ZHANG X Y,REN S.PU-GACNet:Graph Attentionconvolution Network for Point Cloud Upsampling[J].Image and Vision Computing,2022,118:104371.
[7]ALEXA M,BEHR J,COHEN-OR D,et al.Computing and rendering point set surfaces[J]IEEE Transactions on Visualization and Computer Graphics,2003,9(1):3-15.
[8]FERSTL D,REINBACHER C,RANFTL R,et al.Image guided depth upsampling using anisotropic total generalized variation[C]//Proceedings of IEEE International Conference on Computer Vision.2013:993-1000.
[9]SCHUON S,THEOBALT C,DAVIS J,et al.High-quality scan-ning using time-of-flight depth super resolution[C]//2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.2008:1-7.
[10]QI C R,SU H,MO K C,et al.Pointnet:Deep learning on point sets for 3d classification and segmentation[C]//Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition.2017:652-660.
[11]QI C R,LI Y,HAO S,et al.PointNet++:Deep Hierarchical Feature Learning on Point sets in a metric space[C]//Procee-dings of the 31st International Conference on Neural Information Processing Systems.2017:5105-5114.
[12]LI R H,LI X Z,FU C W,et al.PU-GAN:a point cloud upsampling adversarial network[C]//Proceedings of IEEE/CVF International Conference on Computer Vision.2019:7203-7212.
[13]LI R H,LI X Z,HENG P A,et al.Point cloud upsampling viadisentangled refinement[C]//Proceedings of IEEE/CVF Conference on Computer Vision andPattern Recognition.2021:344-353.
[14]LIU H,YUAN H,HOU J,et al.PUFA-GAN:A Frequency-Aware Generative Adversarial Network for 3D point Cloud Upsampling[J].IEEE Transactions on Image Processing,2022,31:7389-7402.
[15]LIU H,YUAN H,HAMZAOUI R,et al.PU-Refiner:A Geo-metry Refiner with Adversarial Learning for Point Cloud Upsampling[C]//IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP 2022).2022:2270-2274.
[16]LUO L Q,TANG L L,ZHOU W Y,et al.PU-EVA:an edge-vector based approximation solution for flexible-scale point cloud upsampling[C]//Proceedings of IEEE/CVF International Conference on Computer Vision.2021:16208-16217.
[17]ZHOU J,CUI G Q,HU S D,et al.Graph neural networks:a review of methods and applications[J].AI Open,2020,1:57-81.
[18]XU Y F,FAN T Q,XU M Y,et al.SpiderCNN:deep learning on point sets with parameterized convolutional filters[C]//Proceedings of European Conference on Computer Vision(ECCV).2018:87-102.
[19]MAO J,WANG X G,LI H S.Interpolated convolutional networksfor 3d point cloud understanding[C]//Proceedings of IEEE/CVF International Conference onComputer Vision.2019:1578-1587.
[20]WANG Y,SUN Y B,LIU Z W,et al.Dynamic graph cnn for learning on point clouds[J].ACM Transactions on Graphics(ToG),2019,38(5):1-12.
[21]LI G H,MÜLLER M,THABET A,et al.DeepGCNs:Can GCNs go as deep as CNNs?[C]//Proceedings of IEEE/CVF International Conference on Computer Vision.2019:9267-9276.
[22]KAZI A,SHEKARFOROUSH S,KRISHNA S A,et al.InceptionGCN:Receptive field aware graph convolutional network for disease prediction[C]//Processingsof 26th International Confe-rence on Medical Imaging.2019:73-85.
[23]WANG L,HUANG Y C,HOU Y L,et al.Graph attention convolution for point cloud semantic segmentation[C]//Procee-dings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:10296-10305.
[24]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[C]//Proceedings of the 31st International Confe-rence on Neural Information Processing Systems.2017:6000-6010.
[25]ZHONG F,BAI Z Y.3D point cloud super resolution with dynamic residual graph convolution networks[J].Journal of Zhejiang University(Engineering Science),2022,56(11):2251-2259.
[26]LIU X H,LIU X C,LIU Y S,et al.SPU-Net:Self-Supervised Point Cloud Upsampling by Coarse-to-Fine Reconstruction With Self-Projection Optimization[J].IEEE Transactions on Image Processing,2022,31:4213-4226.
[27]CHANG A X,FUNKHOUSER T,GUIBAS L,et al.ShapeNet:An Information-Rich 3D ModelRepository[J].arXiv:1512.03012,2015.
[28]WIEMANN T,MITSCHKE I,MOCK A,et al.Surface Reconstruction from Arbitrarily Large Point Clouds[C]//2018 Second IEEE International Conference on Robotic Computing(IRC).2018:278-281.
[29]HOPPE H.Poisson surface reconstruction and its applications[C]//Solid and Physical Modeling.2008.
[1] SUN Yumo, LI Xinhang, ZHAO Wenjie, ZHU Li, LIANG Ya’nan. Driving Towards Intelligent Future:The Application of Deep Learning in Rail Transit Innovation [J]. Computer Science, 2024, 51(8): 1-10.
[2] KONG Lingchao, LIU Guozhu. Review of Outlier Detection Algorithms [J]. Computer Science, 2024, 51(8): 20-33.
[3] LIU Sichun, WANG Xiaoping, PEI Xilong, LUO Hangyu. Scene Segmentation Model Based on Dual Learning [J]. Computer Science, 2024, 51(8): 133-142.
[4] TANG Ruiqi, XIAO Ting, CHI Ziqiu, WANG Zhe. Few-shot Image Classification Based on Pseudo-label Dependence Enhancement and NoiseInterferenceReduction [J]. Computer Science, 2024, 51(8): 152-159.
[5] ZHANG Rui, WANG Ziqi, LI Yang, WANG Jiabao, CHEN Yao. Task-aware Few-shot SAR Image Classification Method Based on Multi-scale Attention Mechanism [J]. Computer Science, 2024, 51(8): 160-167.
[6] WANG Qian, HE Lang, WANG Zhanqing, HUANG Kun. Road Extraction Algorithm for Remote Sensing Images Based on Improved DeepLabv3+ [J]. Computer Science, 2024, 51(8): 168-175.
[7] PU Bin, LIANG Zhengyou, SUN Yu. Monocular 3D Object Detection Based on Height-Depth Constraint and Edge Fusion [J]. Computer Science, 2024, 51(8): 192-199.
[8] ZHANG Junsan, CHENG Ming, SHEN Xiuxuan, LIU Yuxue, WANG Leiquan. Diversified Label Matrix Based Medical Image Report Generation [J]. Computer Science, 2024, 51(8): 200-208.
[9] GUO Fangyuan, JI Genlin. Video Anomaly Detection Method Based on Dual Discriminators and Pseudo Video Generation [J]. Computer Science, 2024, 51(8): 217-223.
[10] WANG Chao, TANG Chao, WANG Wenjian, ZHANG Jing. Infrared Human Action Recognition Method Based on Multimodal Attention Network [J]. Computer Science, 2024, 51(8): 232-241.
[11] ZHANG Lu, DUAN Youxiang, LIU Juan, LU Yuxi. Chinese Geological Entity Relation Extraction Based on RoBERTa and Weighted Graph Convolutional Networks [J]. Computer Science, 2024, 51(8): 297-303.
[12] CHEN Shanshan, YAO Subin. Study on Recommendation Algorithms Based on Knowledge Graph and Neighbor PerceptionAttention Mechanism [J]. Computer Science, 2024, 51(8): 313-323.
[13] CHEN Siyu, MA Hailong, ZHANG Jianhui. Encrypted Traffic Classification of CNN and BiGRU Based on Self-attention [J]. Computer Science, 2024, 51(8): 396-402.
[14] YANG Heng, LIU Qinrang, FAN Wang, PEI Xue, WEI Shuai, WANG Xuan. Study on Deep Learning Automatic Scheduling Optimization Based on Feature Importance [J]. Computer Science, 2024, 51(7): 22-28.
[15] BAI Wenchao, BAI Shuwen, HAN Xixian, ZHAO Yubo. Efficient Query Workload Prediction Algorithm Based on TCN-A [J]. Computer Science, 2024, 51(7): 71-79.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.