Computer Science

Computer Science ›› 2019, Vol. 46 ›› Issue (9): 22-27.doi: 10.11896/j.issn.1002-137X.2019.09.003

• Surverys • Previous Articles     Next Articles

Survey on Character Motion Synthesis Based on Neural Network

WANG Xin1,2, MENG Hao-hao1,2, JIANG Xiao-tao1,2, CHEN Sheng-yong1,3, SUN Ling-yun4,5   

  1. (College of Computer Science and Technology,Zhejiang University of Technology,Hangzhou 310023,China)1;
    (Key Laboratory of Visual Media Intelligent Process Technology of Zhejiang Province,Hangzhou 310023,China)2;
    (College of Computer Science and Engineering,Tianjin University of Technology,Tianjin 300384,China)3;
    (Modern Industrial Design Institute,Zhejiang University,Hangzhou 310058,China)4;
    (International Design Institute,Zhejiang University,Hangzhou 310058,China)5
  • Received:2018-11-14 Online:2019-09-15 Published:2019-09-02

PDF (PC)

1534

Abstract: The application of neural network technology to character motion synthesis on human motion data sets is an important research content in the field of computer graphics.This study aims to generate naturally realistic character motion using neural networks through date-driven technology.Based on the analysis and summary of related research work,this paper introduced the research progress in the fields of motion model construction,motion interaction and motion stylization and so on.Based on the motion capture data,by using data-driven technology,interactive control methods and network models such as ERD,CAE and MAR,the character was dynamically modeled,synthesized and controlled by interactive motion,and in order to generate higher quality character motions,motion animation and other content were stylized.In this paper,taking neural network technology as the focal point,various study works of the character motion synthesis were connected.Combined with the practical applications and difficulties faced in the current research work,this paper suggested some problems that can be further studied.

Key words: Character motion synthesis, Data driven, Interactive character control, Neural network, Style editting

CLC Number: 

  • TP311
[1]GUO S,SOUTHERN R,CHANG J,et al.Adaptive motion synthesis for virtual characters:a survey[J].The Visual Computer,2015,31(5):497-512.
[2]WANG X,CHEN Q,WANG W.3D Human Motion Editing and Synthesis:A Survey[J].Computational and Mathematical Methods in Medicine,2014,2014(3):104535.
[3]LIU G D,PANG Z G,CHENG X,et al.A Survey on Machine Learning in the Synthesis of Human Motions[J].Journal of Computer-Aided Design and Computer Graphics,2010,22(9):1619-1627.(in Chinese)刘更代,潘志庚,程熙,等.人体运动合成中的机器学习技术综述[J].计算机辅助设计与图形学学报,2010,22(9):1619-1627.
[4]KRIZHEVSKY A,SUTSKEVER I,HINTON G E.Imagenetclassification with deep convolutional neural networks[J].Advances in Neural Information Processing Systems,2012,25(2):1097-1105.
[5]SHAZEER N,MIRHOSEINI A,MAZIARZ K,et al.Outra-geously large neural networks:The sparsely-gated mixture-of-experts layer[J].arXiv:1701.06538,2017.
[6]PENG X B,ABBEEL P,LEVINE S,et al.DeepMimic:Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills[J].arXiv:1804.02717,2018.
[7]GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al.Generative adversarial nets[C]//International Conference on Neural Information Processing Systems.MIT Press,Montreal,Canada,2014:2672-2680.
[8]GENG J,SHAO T,ZHENG Y,et al.Warp-guided GANs for single-photo facial animation[C]//SIGGRAPH Asia 2018 Technical Papers.ACM,Tokyo,Japan,2018:231.
[9]KELLY T,GUERRERO P,STEED A,et al.FrankenGAN:Guided Detail Synthesis for Building Mass-Models Using Style-Synchonized GANs[J].arXiv:1806.07179,2018.
[10]BENGIO E,BACON P L,PINEAU J,et al.Conditional computation in neural networks for faster models[J].arXiv:1511.06297,2015.
[11]FU J,CO-REYES J,LEVINE S.Ex2:Exploration with exem-plar models for deep reinforcement learning[C]//Advances in Neural Information Processing Systems.Long Beach,USA,2017:2577-2587.
[12]ZHAO J J,WEI Y,XIA S H,et al.Survey of Physics-BasedCharacter Animation [J].Journal of Computer Research and Development,2015,52(12):2866-2878.(in Chinese)赵建军,魏毅,夏时洪,等.基于物理的角色动画合成方法综述[J].计算机研究与发展,2015,52(12):2866-2878.
[13]ALLEN B F,FALOUTSOS P.Evolved Controllers for Simulated Locomotion[C]//International Workshop on Motion in Games.Springer-Verlag,2009:219-230.
[14]TAN J,GU Y,LIU C K,et al.Learning bicycle stunts[J].Acm Transactions on Graphics,2014,33(4):50.
[15]MORDATCH I,LOWREY K,ANDREW G,et al.Interactive control of diverse complex characters with neural networks[C]//Advances in Neural Information Processing Systems.Long Beach,USA,2015:3132-3140.
[16]LEVINE S,KOLTUN V.Learning Complex Neural Network Policies with Trajectory Optimization[C]//International Conference on Machine Learning.Beijing,China,2014:829-837.
[17]DU Y,WANG W,WANG L.Hierarchical recurrent neural network for skeleton based action recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Boston,USA,2015:1110-1118.
[18]HOLDEN D,SAITO J,KOMURA T,et al.Learning motionmanifolds with convolutional autoencoders[C]//SIGGRAPH Asia 2015 Technical Briefs.Kobe,Japan,2015:18.
[19]HOLDEN D,KOMURA T,SAITO J.Phase-functioned neural networks for character control[J].Acm Transactions on Gra-phics,2017,36(4):1-13.
[20]HOLDEN D,SAITO J,KOMURA T.A deep learning framework for character motion synthesis and editing[J].Acm Transactions on Graphics,2016,35(4):1-11.
[21]XIA S H,WEI Y,WANG Z Q.A Survey of Physics-Based Human Motion Simulation[J].Journal of Computer Research and Development,2010,47(8):1354-1361.(in Chinese)夏时洪,魏毅,王兆其.人体运动仿真综述[J].计算机研究与发展,2010,47(8):1354-1361.
[22]MITTELMAN R,KUIPERS B,SAVARESE S,et al.Structuredrecurrent temporal restricted boltzmann machines[C]//International Conference on Machine Learning.JMLR.org,2014:II-1647.
[23]FRAGKIADAKI K,LEVINE S,FELSEN P,et al.Recurrentnetwork models for human dynamics[C]//Proceedings of the IEEE International Conference on Computer Vision.San Diego,USA,2015:4346-4354.
[24]SAITO S,HU L,MA C,et al.3D hair synthesis using volumetric variational autoencoders[C]//SIGGRAPH Asia 2018 Technical Papers.Tokyo,Japan,2018:208.
[25]CHU M,THUEREY N.Data-driven synthesis of smoke flows with CNN-based feature descriptors[J].ACM Transactions on Graphics (TOG),2017,36(4):69.
[26]XIA S,WANG C,CHAI J,et al.Realtime style transfer for unlabeled heterogeneous human motion[J].ACM Transactions on Graphics,2015,34(4):1-10.
[27]LI Z,ZHOU Y,XIAO S,et al.Auto-conditioned recurrent networks for extended complex human motion synthesis[J].arXiv:1707.05363,2017.
[28]BARSOUM E,KENDER J,LIU Z.HP-GAN:Probabilistic 3Dhuman motion prediction via GAN[J].arXiv:1711.09561,2017.
[29]RAJESWARAN A,KUMAR V,GUPTA A,et al.Learningcomplex dexterous manipulation with deep reinforcementlear-ning and demonstrations[J].arXiv:1709.10087,2017.
[30]MA P,TIAN Y,PAN Z,et al.Fluid directed rigid body control using deep reinforcement learning[J].ACM Transactions on Graphics (TOG),2018,37(4):96.
[31]HUANG T C,HUANG Y J,LIN W C.Real-time horse gaitsynthesis[J].Computer Animation and Virtual Worlds,2013,24(2):87-95.
[32]ZHANG H,STARKE S,KOMURA T,et al.Mode-adaptiveneural networks for quadruped motion control[J].ACM Tran-sactions on Graphics (TOG),2018,37(4):145.
[33]LEE K,LEE S,LEE J.Interactive character animation by lear-ning multi-objective control[C]//SIGGRAPH Asia 2018 Technical Papers.Tokyo,Japan,2018:180.
[34]HO J,ERMON S.Generative adversarial imitation learning[C]//Advances in Neural Information Processing Systems.Barcelona,Spain,2016:4565-4573.
[35]LIU L,HODGINS J.Learning to schedule control fragments for physics-based characters using deep q-learning[J].ACM Transactions on Graphics (TOG),2017,36(3):29.
[36]LAU M,KUFFNER J J.Behavior planning for character animation[C]//ACM Siggraph/eurographics Symposium on ComputerAnimation.Los Angeles,USA,2005:271-280.
[37]SAFONOVA A,HODGINS J K.Construction and optimalsearch of interpolated motion graphs[J].ACM Transactions on Graphics,2007,26(3):106.
[38]LIU L,YIN K K,VAN DE PANNE M,et al.Sampling-based contact-rich motion control[C]//ACM Transactions on Gra-phics (TOG).New York,USA,2010,29(4):128.
[39]MIN J,CHAI J.Motion graphs++:a compact generative model for semantic motion analysis and synthesis[J].ACM Transactions on Graphics (TOG),2012,31(6):153.
[40]HWANG J,KIM J,SUH I H,et al.Real-time Locomotion Controller using an Inverted-Pendulum-based Abstract Model[J].Computer Graphics Forum,2018,37(2):287-296.
[41]RAJESWARAN A,GHOTRA S,RAVINDRAN B,et al.Epopt:Learning robust neural network policies using model ensembles[J].arXiv:1610.01283,2016.
[42]SU W,DU D,YANG X,et al.Interactive sketch-based normal map generation with deep neural networks[J].Proceedings of the ACM on Computer Graphics and Interactive Techniques,2018,1(1):22.
[43]LEVINE S,WANG J M,HARAUX A,et al.Continuous character control with low-dimensional embeddings[J].ACM Tran-sactions on Graphics (TOG),2012,31(4):28.
[44]CLEGG A,YU W,TAN J,et al.Learning to dress:synthesizing human dressing motion via deep reinforcement learning[C]//SIGGRAPH Asia 2018 Technical Papers.Tokyo,Japan,2018:179.
[45]PENG X B,BERSETH G,VAN DE PANNE M.Terrain-adaptive locomotion skills using deep reinforcement learning[J].ACM Transactions on Graphics (TOG),2016,35(4):81.
[46]LIU L,HODGINS J.Learning basketball dribbling skills using trajectory optimization and deep reinforcement learning[J].ACM Transactions on Graphics (TOG),2018,37(4):142.
[47]NAKADA M,CHEN H,TERZOPOULOS D.Learning Biomi-metic Perception for Human Sensorimotor Control[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops.Salt Lake City,USA,2018:1917-1922.
[48]NAKADA M,ZHOU T,CHEN H,et al.Deep learning of biomimetic sensorimotor control for biomechanical human animation[J].ACM Transactions on Graphics (TOG),2018,37(4):56.
[49]NAIR A,MCGREW B,ANDRYCHOWICZ M,et al.Overco-ming exploration in reinforcement learning with demonstrations[C]//2018 IEEE International Conference on Robotics and Automation (ICRA).IEEE,Brisbane,Australis,2018:6292-6299.
[50]GUPTA A,SATKIN S,EFROS A A,et al.From 3d scene geometry to human workspace[C]//2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE,2011:1961-1968.
[51]KIM V G,CHAUDHURI S,GUIBAS L,et al.Shape2pose:Human-centric shape analysis[J].ACM Transactions on Graphics (TOG),2014,33(4):120.
[52]SAVVA M,CHANG A X,Hanrahan P,et al.PiGraphs:learning interaction snapshots from observations[J].ACM Transactions on Graphics (TOG),2016,35(4):139.
[53]KAPADIA M,XU X H,NITTI M,et al.Precision:Precompu-ting environment semantics for contact-rich character animation[C]//Proceedings of the 20th ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games.ACM,2016:29-37.
[54]CHANG X,HOSPEDALES T M,XIANG T.Multi-level factorisation net for person re-identification[C]//CVPR.Salt Lake City,USA,2018:2.
[55]KULKARNI T D,WHITNEY W F,KOHLI P,et al.Deep convolutional inverse graphics network[C]//Advances in Neural Information Processing Systems.Long Beach,USA,2015:2539-2547.
[56]CHOI B,LEWIS J P,SEOL Y,et al.SketchiMo:sketch-basedmotion editing for articulated characters[J].ACM Transactions on Graphics (TOG),2016,35(4):146.
[57]NEFF M,FIUME E.Aesthetic edits for character animation[C]//Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation.San Diego,California:ACM,2003.
[58]COLEMAN P,BIBLIOWICZ J,SINGH K,et al.Staggered poses:a character motion representation for detail-preserving editing of pose and coordinated timing[C]//Eurographics/ACM SIGGRAPH Symposium on Computer Animation,SCA 2008.Dublin,Ireland,DBLP,2008:137-146.
[59]GUAY M,GLEICHER M,CANI M P.Adding dynamics tosketch-based character animations[C]//Sketch-Based Interfaces and Modeling.2015:27-34.
[60]IKEMOTO L,ARIKAN O,FORSYTH D.Generalizing motion edits with Gaussian processes[J].Acm Transactions on Graphi-cs,2009,28(1):1-12.
[61]HSU E,PULLI K.Style translation for human motion[J].ACM Transactions on Graphics (TOG),2005,24(3):1082-1089.
[62]GATYS L A,ECKER S,BETHGE M.A neural algorithm of artistic style[J].arXiv:1508.06576,2015.
[63]GATYS L A,ECKER A S,BETHGE M.Image style transfer using convolutional neural networks[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Las Vegas,USA,2016:2414-2423.
[64]BRAND M,HERTZMANN A.Style machines[C]//Procee-dings of the 27th annual conference on Computer graphics and interactive techniques.ACM Press/Addison-Wesley Publishing Co.,2000:183-192.
[65]MIN J,LIU H,CHAI J.Synthesis and editing of personalizedstylistic human motion[C]//Symposium on Interactive 3d Graphics(Si3d 2010).Washington,DC,USA,DBLP,2010:39-46.
[66]HOLDEN D,HABIBIE I,KUSAJIMA I,et al.Fast NeuralStyle Transfer for Motion Data[J].IEEE computer graphics and applications,2017,37(4):42-49.
[1] ZHOU Fang-quan, CHENG Wei-qing. Sequence Recommendation Based on Global Enhanced Graph Neural Network [J]. Computer Science, 2022, 49(9): 55-63.
[2] ZHOU Le-yuan, ZHANG Jian-hua, YUAN Tian-tian, CHEN Sheng-yong. Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion [J]. Computer Science, 2022, 49(9): 155-161.
[3] NING Han-yang, MA Miao, YANG Bo, LIU Shi-chang. Research Progress and Analysis on Intelligent Cryptology [J]. Computer Science, 2022, 49(9): 288-296.
[4] WANG Run-an, ZOU Zhao-nian. Query Performance Prediction Based on Physical Operation-level Models [J]. Computer Science, 2022, 49(8): 49-55.
[5] CHEN Yong-quan, JIANG Ying. Analysis Method of APP User Behavior Based on Convolutional Neural Network [J]. Computer Science, 2022, 49(8): 78-85.
[6] ZHU Cheng-zhang, HUANG Jia-er, XIAO Ya-long, WANG Han, ZOU Bei-ji. Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism [J]. Computer Science, 2022, 49(8): 113-119.
[7] YAN Jia-dan, JIA Cai-yan. Text Classification Method Based on Information Fusion of Dual-graph Neural Network [J]. Computer Science, 2022, 49(8): 230-236.
[8] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[9] QI Xiu-xiu, WANG Jia-hao, LI Wen-xiong, ZHOU Fan. Fusion Algorithm for Matrix Completion Prediction Based on Probabilistic Meta-learning [J]. Computer Science, 2022, 49(7): 18-24.
[10] YANG Bing-xin, GUO Yan-rong, HAO Shi-jie, Hong Ri-chang. Application of Graph Neural Network Based on Data Augmentation and Model Ensemble in Depression Recognition [J]. Computer Science, 2022, 49(7): 57-63.
[11] ZHANG Ying-tao, ZHANG Jie, ZHANG Rui, ZHANG Wen-qiang. Photorealistic Style Transfer Guided by Global Information [J]. Computer Science, 2022, 49(7): 100-105.
[12] DAI Zhao-xia, LI Jin-xin, ZHANG Xiang-dong, XU Xu, MEI Lin, ZHANG Liang. Super-resolution Reconstruction of MRI Based on DNGAN [J]. Computer Science, 2022, 49(7): 113-119.
[13] LIU Yue-hong, NIU Shao-hua, SHEN Xian-hao. Virtual Reality Video Intraframe Prediction Coding Based on Convolutional Neural Network [J]. Computer Science, 2022, 49(7): 127-131.
[14] XU Ming-ke, ZHANG Fan. Head Fusion:A Method to Improve Accuracy and Robustness of Speech Emotion Recognition [J]. Computer Science, 2022, 49(7): 132-141.
[15] PENG Shuang, WU Jiang-jiang, CHEN Hao, DU Chun, LI Jun. Satellite Onboard Observation Task Planning Based on Attention Neural Network [J]. Computer Science, 2022, 49(7): 242-247.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.