计算机科学 ›› 2021, Vol. 48 ›› Issue (4): 192-196.doi: 10.11896/jsjkx.200100048

• 计算机图形学&多媒体 • 上一篇    下一篇

3D点云形状补全GAN

赵新灿, 常寒星, 金仁标   

  1. 郑州大学信息工程学院 郑州450001
  • 收稿日期:2020-06-24 修回日期:2020-06-03 出版日期:2021-04-15 发布日期:2021-04-09
  • 通讯作者: 赵新灿(iexczhao@zzu.edu.cn)
  • 基金资助:
    航空科学基金(2018ZC41002)

3D Point Cloud Shape Completion GAN

ZHAO Xin-can, CHANG Han-xing, JIN Ren-biao   

  1. School of Information Engineering,Zhengzhou University,Zhengzhou 450001,China
  • Received:2020-06-24 Revised:2020-06-03 Online:2021-04-15 Published:2021-04-09
  • About author:ZHAO Xin-can,born in 1972,Ph.D,associate professor.His main research interests include virtual reality(VR) and augmented reality(AR).
  • Supported by:
    Aeronautical Science Foundation of China (2018ZC41002).

摘要: 在真实的扫描环境中,由于视线遮挡或技术人员操作不当,实际采集到的点云模型会存在形状不完整的问题。点云模型的不完整性会对后续应用产生严重的影响,因此提出3D点云形状补全GAN用于完成点云模型的形状补全。该网络的点云重建部分由PointNet中用于数据对齐的T-Net结构与3D点云AutoEncoder网络相结合,来完成预测和填充缺失数据,识别器采用3D点云AutoEncoder中的Encoder部分对补全3D点云数据与真实的3D点云数据进行区分。最后,在ShapeNet数据集中训练上述网络结构,对所训练的网络模型进行验证并与其他基准方法进行定性比较。从实验结果可以看出,3D点云形状补全GAN可以将具有缺失数据的点云模型补全为完整的3D点云。在ShapeNet的3个子数据集chair,table以及bed上,相比基于3D点云AutoEncoder的方法,所提方法的F1分数分别提高了3.0%,3.3%以及3.1%,相比基于体素3D-EPN的方法,所提方法的F1分数分别提高了9.9%,5.8%以及4.3%。

关键词: 3D 点云, AutoEncoder, 生成对抗网络, 形状补全

Abstract: In the real scanning environment,due to the occlusion of line of sight or improper operation of technicians,the actual point cloud model has incomplete shape.The incompleteness of point cloud model has a serious impact on subsequent applications.Therefore,this paper proposes a 3D point cloud shape completion generative adversarial networks to complete the shape completion of point cloud model.The point cloud reconstruction part of the network combines the T-Net structure used for data alignment in PointNet with the 3D point cloud AutoEncoder network to complete the prediction and fill in the missing data.The discriminator uses the Encoder part of the 3D point cloud AutoEncoder to distinguish the completed 3D point cloud data from the real 3D point cloud data.Finally,in the ShapeNet trained the above network structure,the trained network model is verified and compared with other benchmark methods qualitatively.From the experimental results,it can be seen that the 3D point cloud shape completion generation adversarial network can complete the point cloud model with missing data into a complete 3D point cloud.In ShapeNet’s three sub-datasets chair,table,and bed,compared with the method based on 3D point cloud AutoEncoder,the F1 score is improved by 3.0%,3.3% and 3.1%,and compared with the method based on the voxel 3D-EPN method,the F1 score is increased by 9.9%,5.8%,and 4.3%,respectively.

Key words: 3D point cloud, AutoEncoder, Generative adversarial network, Shape completion

中图分类号: 

  • TP391
[1]LEONG K F,CHUA C K,NG Y M.A study of stereolithography file errors and repair.Part 1.Generic solution[J].The International Journal of Advanced Manufacturing Technology,1996,12(6):407-414.
[2]LEONG K F,CHUA C K,NG Y M.A study of stereolithography file errors and repair.Part 2.Special cases[J].The International Journal of Advanced Manufacturing Technology,1996,12(6):415-422.
[3]PFEIFLE R,SEIDEL H P.Triangular B-splines for Blending & Filling of Polygonal Holes[C]//Graphics Inter-face.1996:186-193.
[4]ISHIGAMI G,OTSUKI M,KUBOTA T.Path planning andnavigation framework for a planetary exploration rover using a laser range finder[C]//Field and Service Robotics.Springer,Berlin,Heidelberg,2014:431-447.
[5]YANG R,FENG Y Q,YUAN X J.Restora-tion of Fragmentary 3D Models Using Existing Models[J].Journal of Com-puter-Aided Design & Computer Gra-phics,2015(1):98-105.
[6]SUNG M,KIM V G,ANGST R,et al.Data-driven structural priors for shape completion[J].ACM Transactions on Graphics (TOG),2015,34(6):1-11.
[7]ROCK J,GUPTA T,THORSEN J,et al.Completing 3d object shape from one depth image[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:2484-2493.
[8]LI D,SHAO T,WU H,et al.Shape com-pletion from a single rgbd image[J].IEEE Transactions on Visualization and Computer Graphics,2016,23(7):1809-1822.
[9]WU J,ZHANG C,XUE T,et al.Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling[C]//Advances in Neural Information Processing Systems.2016:82-90.
[10]WANG W,HUANG Q,YOU S,et al.Shape inpainting using 3d generative adver-sarial network and recurrent convolu-tional networks[C]//Proceedings of the IEEE InternationalConfe-rence on Computer Vision.2017:2298-2306.
[11]YU C,WANG Y.3D-Scene-GAN:Three-dimensional Scene Reconstruction with Generative Adversarial Networks[C]//ICLR (Workshop).2018:1-4.
[12]DAI A,RITCHIE D,BOKELOH M,et al.Scancomplete:Large-scale scene com-pletion and semantic segmentation for 3d scans[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:4578-4587.
[13]QI C R,SU H,MO K,et al.Pointnet:Deep learning on pointsets for 3d classification and segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:652-660.
[14]QI C R,YI L,SU H,et al.Pointnet++:Deep hierarchical feature learning on point sets in a metric space[C]//Advances in Neural Information Processing Systems.2017:5099-5108.
[15]ACHLIOPTAS P,DIAMANTI O,MITLIAGKAS I,et al.Learning representations and generative models for 3d point clouds[J].arXiv:1707.02392.
[16]CHANG A X,FUNKHOUSER T,GUIBAS L,et al.Shapenet:An information-rich 3d model repository[J].arXiv:1512.03012,2015.
[17]ALDOMA A,MARTON Z C,TOMBARI F,et al.Tutorial:Point cloud library:Three-dimensional object recognition and 6 dof pose estimation[J].IEEE Robotics & Automation Magazine,2012,19(3):80-91.
[18]IOFFE S,SZEGEDY C.Batch normaliza-tion:Accelerating deep network training by reducing internal covariate shift[J].arXiv:1502.03167.
[19]CHEN X,CHEN B,MITRA N J.Unpaired Point Cloud Completion on Real Scans using Adversarial Training[J].arXiv:1904.00069,2019.
[20]DAI A,RUIZHONGTAI QI C,NIEβNER M.Shape comple-tion using 3d-encoder-predictor cnns and shape synthesis[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:5868-5877.
[21]YUAN W,KHOT T,HELD D,et al.Pcn:Point completion network[C]//2018 International Conference on 3D Vision (3DV).IEEE,2018:728-737.
[1] 张佳, 董守斌.
基于评论方面级用户偏好迁移的跨领域推荐算法
Cross-domain Recommendation Based on Review Aspect-level User Preference Transfer
计算机科学, 2022, 49(9): 41-47. https://doi.org/10.11896/jsjkx.220200131
[2] 孙奇, 吉根林, 张杰.
基于非局部注意力生成对抗网络的视频异常事件检测方法
Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection
计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[3] 戴朝霞, 李锦欣, 张向东, 徐旭, 梅林, 张亮.
基于DNGAN的磁共振图像超分辨率重建算法
Super-resolution Reconstruction of MRI Based on DNGAN
计算机科学, 2022, 49(7): 113-119. https://doi.org/10.11896/jsjkx.210600105
[4] 尹文兵, 高戈, 曾邦, 王霄, 陈怡.
基于时频域生成对抗网络的语音增强算法
Speech Enhancement Based on Time-Frequency Domain GAN
计算机科学, 2022, 49(6): 187-192. https://doi.org/10.11896/jsjkx.210500114
[5] 徐辉, 康金梦, 张加万.
基于特征感知的数字壁画复原方法
Digital Mural Inpainting Method Based on Feature Perception
计算机科学, 2022, 49(6): 217-223. https://doi.org/10.11896/jsjkx.210500105
[6] 高志宇, 王天荆, 汪悦, 沈航, 白光伟.
基于生成对抗网络的5G网络流量预测方法
Traffic Prediction Method for 5G Network Based on Generative Adversarial Network
计算机科学, 2022, 49(4): 321-328. https://doi.org/10.11896/jsjkx.210300240
[7] 黎思泉, 万永菁, 蒋翠玲.
基于生成对抗网络去影像的多基频估计算法
Multiple Fundamental Frequency Estimation Algorithm Based on Generative Adversarial Networks for Image Removal
计算机科学, 2022, 49(3): 179-184. https://doi.org/10.11896/jsjkx.201200081
[8] 李建, 郭延明, 于天元, 武与伦, 王翔汉, 老松杨.
基于生成对抗网络的多目标类别对抗样本生成算法
Multi-target Category Adversarial Example Generating Algorithm Based on GAN
计算机科学, 2022, 49(2): 83-91. https://doi.org/10.11896/jsjkx.210800130
[9] 谈馨悦, 何小海, 王正勇, 罗晓东, 卿粼波.
基于Transformer交叉注意力的文本生成图像技术
Text-to-Image Generation Technology Based on Transformer Cross Attention
计算机科学, 2022, 49(2): 107-115. https://doi.org/10.11896/jsjkx.210600085
[10] 陈贵强, 何军.
自然场景下遥感图像超分辨率重建算法研究
Study on Super-resolution Reconstruction Algorithm of Remote Sensing Images in Natural Scene
计算机科学, 2022, 49(2): 116-122. https://doi.org/10.11896/jsjkx.210700095
[11] 石达, 芦天亮, 杜彦辉, 张建岭, 暴雨轩.
基于改进CycleGAN的人脸性别伪造图像生成模型
Generation Model of Gender-forged Face Image Based on Improved CycleGAN
计算机科学, 2022, 49(2): 31-39. https://doi.org/10.11896/jsjkx.210600012
[12] 唐雨潇, 王斌君.
基于深度生成模型的人脸编辑研究进展
Research Progress of Face Editing Based on Deep Generative Model
计算机科学, 2022, 49(2): 51-61. https://doi.org/10.11896/jsjkx.210400108
[13] 张玮琪, 汤轶丰, 李林燕, 胡伏原.
基于场景图的段落生成序列图像方法
Image Stream From Paragraph Method Based on Scene Graph
计算机科学, 2022, 49(1): 233-240. https://doi.org/10.11896/jsjkx.201100207
[14] 蒋宗礼, 樊珂, 张津丽.
基于生成对抗网络和元路径的异质网络表示学习
Generative Adversarial Network and Meta-path Based Heterogeneous Network Representation Learning
计算机科学, 2022, 49(1): 133-139. https://doi.org/10.11896/jsjkx.201000179
[15] 徐涛, 田崇阳, 刘才华.
基于深度学习的人群异常行为检测综述
Deep Learning for Abnormal Crowd Behavior Detection:A Review
计算机科学, 2021, 48(9): 125-134. https://doi.org/10.11896/jsjkx.201100015
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!