Computer Science ›› 2022, Vol. 49 ›› Issue (10): 207-213.doi: 10.11896/jsjkx.210900066

• Computer Graphics& Multimedia • Previous Articles     Next Articles

Voxel Deformation Network Based on Environmental Information Mining

LIU Na-li1,3, TIAN Yan1,3, SONG Ya-dong4, JIANG Teng-fei3, WANG Xun1,2, YANG Bai-lin1   

  1. 1 School of Computer Science & Information Engineering,Zhejiang Gongshang University,Hangzhou 310018,China
    2 Zhejiang Lab,Hangzhou 311121,China
    3 Shining 3D Research,Shining 3D Tech Co.,Ltd,Hangzhou 310013,China
    4 School of Information Science Engineering,East China University of Science and Technology,Shanghai 200237,China
  • Received:2021-09-08 Revised:2022-01-16 Online:2022-10-15 Published:2022-10-13
  • About author:LIU Na-li,born in 1996,postgraduate,is a member of China Computer Federation.Her main research interest include computer vision and deep learning.
    TIAN Yan,born in 1982,Ph.D,associate professor,master supervisor,is a member of China Computer Federation.His main research interests include machine learning and video analysis.
  • Supported by:
    National Key R&D Program of China(2018YFB1404102,2018YFB1403200),National Natural Science Foundation of China(61972351,61976188,61972353),National Natural Science Foundation of Zhejiang Province(LY19F030005),Opening Foundation of State Key Laboratory of Virtual Reality Technology and System of Beihang University,China(VRLAB2020B15) and Zhejiang Laboratory Funded Project(2019KD0AC02).

Abstract: The technique of 3D deformation is one of the hot topics in the field of computer graphics.Current 3D deformation methods mainly learn the changes before and after deformation by aggregating localized adjacent voxel features,and fail to exploit the interrelationship between non-local voxel features,and the absence of contextual information prevents the model from capturing more discriminative features.To address the above problems,this paper designs a voxel deformation network based on environmental information mining,which can extract local and environmental information simultaneously,and extract environmental information from different spatial domains to improve the representation performance of the network,further modeling the relationship before and after the deformation of the object.Firstly,a novel self-attention mechanism is introduced.Specifically,the learning of the non-local dependence of different voxels is proposed to improve the ability of voxel discrimination.Then,a multi-scale analysis method is introduced to extract environmental information in different perceptual fields via multiple dilated convolution with different dilation rates,which provides more informative contextual features for the subsequent models.In addition,this paper analyzes the impact of feature fusion on the model and designs a method based on encoder-decoder feature fusion,which adaptively fuses the features extracted from the encoder and decoder to improve the nonlinear mapping capability of the model.Extensive experiments are conducted on our tooth dataset.The results show that the deformation prediction accuracy of the proposed method is improved compared to existing methods.

Key words: Shape deformation, Voxel, Attention mechanism, Feature fusion, Multiscale analysis

CLC Number: 

  • TP391
[1]WANG W,CEYLAN D,MECH R,et al.3DN:3D Deformation Network[C]//Proceedings of the IEEE Conference on Compu-ter Vision and Pattern Recognition.Long Beach:IEEE Press,2019:1038-1046.
[2]YUMER M E,MITRA N J.Learning Semantic DeformationFlows with 3D Convolutional Networks[C]//Proceedings of the European Conference on Computer Vision.Amsterdam:Springer International Publishing Press,2016:294-311.
[3]SUMNER R W,POPVIC J.Deformation Transfer for Triangle Meshes [J].ACM Transactions on Graphics,2004,23(3):399-405.
[4]CHU H K,LIN C H.Example-based Deformation Transfer for 3D Polygon Models[J].Journal of Information Science and Engineering,2010,26(2):379-391.
[5]GROUEIX T,FISHER M,KIM V G,et al.3D-CODED:3D Correspondences by Deep Deformation[C]//Proceedings of the European Conference on Computer Vision.Munich:Springer International Publishing Press,2018:230-246.
[6]YE Y T.Research of Deep Neural Network Based Face Guided Image Completion[D].Harbin:Harbin Institute of Technology.2018.
[7]RU X Q,HUA G G,LI L H,et al.Handwritten Digital Recognition Based on Deformable Convolutional Neural Network[J].Microeletronics&Computer,2019,36(4):47-51.
[8]QI C R,SU H,MO K,et al.PointNet:Deep Learning on Point Sets for 3D Classification and Segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Honolulu:IEEE Computer Society Press,2017:77-85.
[9]TIAN Y,CHEN T,CHENG G,et al.Global Context Assisted Structure-Aware Vehicle Retrieval[J].IEEE Transactions on Intelligent Transportation Systems,2020,23(1):165-174.
[10]TIAN Y,CHENG G,GELERNTER J,et al.Joint TemporalContext exploitation and Active Learning for Video segmentation[J].Pattern Recognition,2020,100:107158.
[11]TIAN Y,ZHANG Y,ZHOU D,et al.Triple Attention Network For video Segmentation[J].Neuro Computing,2020,417:202-211.
[12]TIAN Y,GELERNTER J,WANG X,et al.Traffic Sign Detection Using a Multi-Scale Recurrent Attention Network[J].IEEE Transactions on Intelligent Transportation Systems,2019,20(12):565-571.
[13]MILLETARI F,NAVAB N,AHMADI S A.V-Net:Fully Convo-lutional Neural Networks for Volumetric Medical Image Segmentation[C]//Proceedings of Fourth International Conference on 3D Vision.Stanford:IEEE Computer Society,2016:565-571.
[14]NOH J,NEUMANN U.Expression Cloning[C]//Proceedings of the 28td Annual Conference on Computer Graphics and Interactive Techniques.New York:Association for Computing Machinery Press,2001:277-288.
[15]TAN Q,GAO L,LAI Y K,et al.Mesh-Based Autoencoders for Localized Deformation Component Analysis[C]//Proceedings of the AAAI Conference on Artificial Intelligence.New Orleans:AAAI Press.2018:2452-2459.
[16]GAO L,YANG J,QIAO Y L,et al.Automatic UnpairedShape Deformation Transfer[J].ACM Transactions on Gra-phics,2018,237:1-15.
[17]RONNEBERGER O,FISCHER P,BROX T.U-Net:Convolu-tional Networks for Biomedical Image Segmentation[C]//International Conference on Medical Image Computing and Compu-ter-Assisted Intervention.Munich:Springer International Publishing Press,2015:234-241.
[18]WANG X,GIRSHICK R,GUPTA A,et al.Non-local NeuralNetworks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Salt Lake:IEEE Press,2018:7794-7803.
[19]WERNER D,AL-HAMADI A,WERNER P.Truncated Signed Distance Function:Experiments on Voxel Size[C]//Interna-tional Conference Image Analysis and Recognition.Algarve:Springer International Publishing Press,2014:357-364.
[20]YIN K,CHEN Z,HUANG H,et al.LOGAN:Unpaired Shape Transform in Latent Overcomplete Space[J].ACM Transactions on Graphics,2019,38(6):1-13.
[21]SHIMADA S,GOLYANIK V,TRETSCHK E,et al.Dispvoxnets:Non-Rigid Point Set Alignment with Supervised Learning Proxies[C]//2019 International Conference on 3D Vision.Cana-da:IEEE Press,2019:27-36.
[1] ZHOU Fang-quan, CHENG Wei-qing. Sequence Recommendation Based on Global Enhanced Graph Neural Network [J]. Computer Science, 2022, 49(9): 55-63.
[2] DAI Yu, XU Lin-feng. Cross-image Text Reading Method Based on Text Line Matching [J]. Computer Science, 2022, 49(9): 139-145.
[3] ZHOU Le-yuan, ZHANG Jian-hua, YUAN Tian-tian, CHEN Sheng-yong. Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion [J]. Computer Science, 2022, 49(9): 155-161.
[4] XIONG Li-qin, CAO Lei, LAI Jun, CHEN Xi-liang. Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization [J]. Computer Science, 2022, 49(9): 172-182.
[5] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[6] ZHU Cheng-zhang, HUANG Jia-er, XIAO Ya-long, WANG Han, ZOU Bei-ji. Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism [J]. Computer Science, 2022, 49(8): 113-119.
[7] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[8] YAN Jia-dan, JIA Cai-yan. Text Classification Method Based on Information Fusion of Dual-graph Neural Network [J]. Computer Science, 2022, 49(8): 230-236.
[9] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[10] WANG Ming, PENG Jian, HUANG Fei-hu. Multi-time Scale Spatial-Temporal Graph Neural Network for Traffic Flow Prediction [J]. Computer Science, 2022, 49(8): 40-48.
[11] JIN Fang-yan, WANG Xiu-li. Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM [J]. Computer Science, 2022, 49(7): 179-186.
[12] XIONG Luo-geng, ZHENG Shang, ZOU Hai-tao, YU Hua-long, GAO Shang. Software Self-admitted Technical Debt Identification with Bidirectional Gate Recurrent Unit and Attention Mechanism [J]. Computer Science, 2022, 49(7): 212-219.
[13] PENG Shuang, WU Jiang-jiang, CHEN Hao, DU Chun, LI Jun. Satellite Onboard Observation Task Planning Based on Attention Neural Network [J]. Computer Science, 2022, 49(7): 242-247.
[14] ZHANG Ying-tao, ZHANG Jie, ZHANG Rui, ZHANG Wen-qiang. Photorealistic Style Transfer Guided by Global Information [J]. Computer Science, 2022, 49(7): 100-105.
[15] ZENG Zhi-xian, CAO Jian-jun, WENG Nian-feng, JIANG Guo-quan, XU Bin. Fine-grained Semantic Association Video-Text Cross-modal Entity Resolution Based on Attention Mechanism [J]. Computer Science, 2022, 49(7): 106-112.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!