计算机科学 ›› 2019, Vol. 46 ›› Issue (9): 22-27.doi: 10.11896/j.issn.1002-137X.2019.09.003
王鑫1,2, 孟浩浩1,2, 姜小涛1,2, 陈胜勇1,3, 孙凌云4,5
WANG Xin1,2, MENG Hao-hao1,2, JIANG Xiao-tao1,2, CHEN Sheng-yong1,3, SUN Ling-yun4,5
摘要: 在角色运动数据集上,运用神经网络技术进行运动合成是当前计算机图形学领域中的一项重要研究。该研究旨在通过神经网络技术生成自然、逼真度较高的角色运动。在对相关研究工作进行分析和总结的基础上,对运动模型的构建、运动交互和运动风格化等领域的研究进展进行了介绍;详细阐述了基于运动捕获数据,利用数据驱动技术、交互式控制方法和ERD,CAE,MAR等网络模型,动态地对角色进行运动建模、运动合成、交互式运动控制,同时为了合成更高质量的角色运动,对运动动画进行风格化等处理;以神经网络技术为着眼点,串联角色运动合成中的各个环节,并结合实际应用,针对当前研究工作面临的难点提出一些可继续深入探索的问题。
中图分类号:
[1]GUO S,SOUTHERN R,CHANG J,et al.Adaptive motion synthesis for virtual characters:a survey[J].The Visual Computer,2015,31(5):497-512. [2]WANG X,CHEN Q,WANG W.3D Human Motion Editing and Synthesis:A Survey[J].Computational and Mathematical Methods in Medicine,2014,2014(3):104535. [3]LIU G D,PANG Z G,CHENG X,et al.A Survey on Machine Learning in the Synthesis of Human Motions[J].Journal of Computer-Aided Design and Computer Graphics,2010,22(9):1619-1627.(in Chinese)刘更代,潘志庚,程熙,等.人体运动合成中的机器学习技术综述[J].计算机辅助设计与图形学学报,2010,22(9):1619-1627. [4]KRIZHEVSKY A,SUTSKEVER I,HINTON G E.Imagenetclassification with deep convolutional neural networks[J].Advances in Neural Information Processing Systems,2012,25(2):1097-1105. [5]SHAZEER N,MIRHOSEINI A,MAZIARZ K,et al.Outra-geously large neural networks:The sparsely-gated mixture-of-experts layer[J].arXiv:1701.06538,2017. [6]PENG X B,ABBEEL P,LEVINE S,et al.DeepMimic:Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills[J].arXiv:1804.02717,2018. [7]GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al.Generative adversarial nets[C]//International Conference on Neural Information Processing Systems.MIT Press,Montreal,Canada,2014:2672-2680. [8]GENG J,SHAO T,ZHENG Y,et al.Warp-guided GANs for single-photo facial animation[C]//SIGGRAPH Asia 2018 Technical Papers.ACM,Tokyo,Japan,2018:231. [9]KELLY T,GUERRERO P,STEED A,et al.FrankenGAN:Guided Detail Synthesis for Building Mass-Models Using Style-Synchonized GANs[J].arXiv:1806.07179,2018. [10]BENGIO E,BACON P L,PINEAU J,et al.Conditional computation in neural networks for faster models[J].arXiv:1511.06297,2015. [11]FU J,CO-REYES J,LEVINE S.Ex2:Exploration with exem-plar models for deep reinforcement learning[C]//Advances in Neural Information Processing Systems.Long Beach,USA,2017:2577-2587. [12]ZHAO J J,WEI Y,XIA S H,et al.Survey of Physics-BasedCharacter Animation [J].Journal of Computer Research and Development,2015,52(12):2866-2878.(in Chinese)赵建军,魏毅,夏时洪,等.基于物理的角色动画合成方法综述[J].计算机研究与发展,2015,52(12):2866-2878. [13]ALLEN B F,FALOUTSOS P.Evolved Controllers for Simulated Locomotion[C]//International Workshop on Motion in Games.Springer-Verlag,2009:219-230. [14]TAN J,GU Y,LIU C K,et al.Learning bicycle stunts[J].Acm Transactions on Graphics,2014,33(4):50. [15]MORDATCH I,LOWREY K,ANDREW G,et al.Interactive control of diverse complex characters with neural networks[C]//Advances in Neural Information Processing Systems.Long Beach,USA,2015:3132-3140. [16]LEVINE S,KOLTUN V.Learning Complex Neural Network Policies with Trajectory Optimization[C]//International Conference on Machine Learning.Beijing,China,2014:829-837. [17]DU Y,WANG W,WANG L.Hierarchical recurrent neural network for skeleton based action recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Boston,USA,2015:1110-1118. [18]HOLDEN D,SAITO J,KOMURA T,et al.Learning motionmanifolds with convolutional autoencoders[C]//SIGGRAPH Asia 2015 Technical Briefs.Kobe,Japan,2015:18. [19]HOLDEN D,KOMURA T,SAITO J.Phase-functioned neural networks for character control[J].Acm Transactions on Gra-phics,2017,36(4):1-13. [20]HOLDEN D,SAITO J,KOMURA T.A deep learning framework for character motion synthesis and editing[J].Acm Transactions on Graphics,2016,35(4):1-11. [21]XIA S H,WEI Y,WANG Z Q.A Survey of Physics-Based Human Motion Simulation[J].Journal of Computer Research and Development,2010,47(8):1354-1361.(in Chinese)夏时洪,魏毅,王兆其.人体运动仿真综述[J].计算机研究与发展,2010,47(8):1354-1361. [22]MITTELMAN R,KUIPERS B,SAVARESE S,et al.Structuredrecurrent temporal restricted boltzmann machines[C]//International Conference on Machine Learning.JMLR.org,2014:II-1647. [23]FRAGKIADAKI K,LEVINE S,FELSEN P,et al.Recurrentnetwork models for human dynamics[C]//Proceedings of the IEEE International Conference on Computer Vision.San Diego,USA,2015:4346-4354. [24]SAITO S,HU L,MA C,et al.3D hair synthesis using volumetric variational autoencoders[C]//SIGGRAPH Asia 2018 Technical Papers.Tokyo,Japan,2018:208. [25]CHU M,THUEREY N.Data-driven synthesis of smoke flows with CNN-based feature descriptors[J].ACM Transactions on Graphics (TOG),2017,36(4):69. [26]XIA S,WANG C,CHAI J,et al.Realtime style transfer for unlabeled heterogeneous human motion[J].ACM Transactions on Graphics,2015,34(4):1-10. [27]LI Z,ZHOU Y,XIAO S,et al.Auto-conditioned recurrent networks for extended complex human motion synthesis[J].arXiv:1707.05363,2017. [28]BARSOUM E,KENDER J,LIU Z.HP-GAN:Probabilistic 3Dhuman motion prediction via GAN[J].arXiv:1711.09561,2017. [29]RAJESWARAN A,KUMAR V,GUPTA A,et al.Learningcomplex dexterous manipulation with deep reinforcementlear-ning and demonstrations[J].arXiv:1709.10087,2017. [30]MA P,TIAN Y,PAN Z,et al.Fluid directed rigid body control using deep reinforcement learning[J].ACM Transactions on Graphics (TOG),2018,37(4):96. [31]HUANG T C,HUANG Y J,LIN W C.Real-time horse gaitsynthesis[J].Computer Animation and Virtual Worlds,2013,24(2):87-95. [32]ZHANG H,STARKE S,KOMURA T,et al.Mode-adaptiveneural networks for quadruped motion control[J].ACM Tran-sactions on Graphics (TOG),2018,37(4):145. [33]LEE K,LEE S,LEE J.Interactive character animation by lear-ning multi-objective control[C]//SIGGRAPH Asia 2018 Technical Papers.Tokyo,Japan,2018:180. [34]HO J,ERMON S.Generative adversarial imitation learning[C]//Advances in Neural Information Processing Systems.Barcelona,Spain,2016:4565-4573. [35]LIU L,HODGINS J.Learning to schedule control fragments for physics-based characters using deep q-learning[J].ACM Transactions on Graphics (TOG),2017,36(3):29. [36]LAU M,KUFFNER J J.Behavior planning for character animation[C]//ACM Siggraph/eurographics Symposium on ComputerAnimation.Los Angeles,USA,2005:271-280. [37]SAFONOVA A,HODGINS J K.Construction and optimalsearch of interpolated motion graphs[J].ACM Transactions on Graphics,2007,26(3):106. [38]LIU L,YIN K K,VAN DE PANNE M,et al.Sampling-based contact-rich motion control[C]//ACM Transactions on Gra-phics (TOG).New York,USA,2010,29(4):128. [39]MIN J,CHAI J.Motion graphs++:a compact generative model for semantic motion analysis and synthesis[J].ACM Transactions on Graphics (TOG),2012,31(6):153. [40]HWANG J,KIM J,SUH I H,et al.Real-time Locomotion Controller using an Inverted-Pendulum-based Abstract Model[J].Computer Graphics Forum,2018,37(2):287-296. [41]RAJESWARAN A,GHOTRA S,RAVINDRAN B,et al.Epopt:Learning robust neural network policies using model ensembles[J].arXiv:1610.01283,2016. [42]SU W,DU D,YANG X,et al.Interactive sketch-based normal map generation with deep neural networks[J].Proceedings of the ACM on Computer Graphics and Interactive Techniques,2018,1(1):22. [43]LEVINE S,WANG J M,HARAUX A,et al.Continuous character control with low-dimensional embeddings[J].ACM Tran-sactions on Graphics (TOG),2012,31(4):28. [44]CLEGG A,YU W,TAN J,et al.Learning to dress:synthesizing human dressing motion via deep reinforcement learning[C]//SIGGRAPH Asia 2018 Technical Papers.Tokyo,Japan,2018:179. [45]PENG X B,BERSETH G,VAN DE PANNE M.Terrain-adaptive locomotion skills using deep reinforcement learning[J].ACM Transactions on Graphics (TOG),2016,35(4):81. [46]LIU L,HODGINS J.Learning basketball dribbling skills using trajectory optimization and deep reinforcement learning[J].ACM Transactions on Graphics (TOG),2018,37(4):142. [47]NAKADA M,CHEN H,TERZOPOULOS D.Learning Biomi-metic Perception for Human Sensorimotor Control[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops.Salt Lake City,USA,2018:1917-1922. [48]NAKADA M,ZHOU T,CHEN H,et al.Deep learning of biomimetic sensorimotor control for biomechanical human animation[J].ACM Transactions on Graphics (TOG),2018,37(4):56. [49]NAIR A,MCGREW B,ANDRYCHOWICZ M,et al.Overco-ming exploration in reinforcement learning with demonstrations[C]//2018 IEEE International Conference on Robotics and Automation (ICRA).IEEE,Brisbane,Australis,2018:6292-6299. [50]GUPTA A,SATKIN S,EFROS A A,et al.From 3d scene geometry to human workspace[C]//2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE,2011:1961-1968. [51]KIM V G,CHAUDHURI S,GUIBAS L,et al.Shape2pose:Human-centric shape analysis[J].ACM Transactions on Graphics (TOG),2014,33(4):120. [52]SAVVA M,CHANG A X,Hanrahan P,et al.PiGraphs:learning interaction snapshots from observations[J].ACM Transactions on Graphics (TOG),2016,35(4):139. [53]KAPADIA M,XU X H,NITTI M,et al.Precision:Precompu-ting environment semantics for contact-rich character animation[C]//Proceedings of the 20th ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games.ACM,2016:29-37. [54]CHANG X,HOSPEDALES T M,XIANG T.Multi-level factorisation net for person re-identification[C]//CVPR.Salt Lake City,USA,2018:2. [55]KULKARNI T D,WHITNEY W F,KOHLI P,et al.Deep convolutional inverse graphics network[C]//Advances in Neural Information Processing Systems.Long Beach,USA,2015:2539-2547. [56]CHOI B,LEWIS J P,SEOL Y,et al.SketchiMo:sketch-basedmotion editing for articulated characters[J].ACM Transactions on Graphics (TOG),2016,35(4):146. [57]NEFF M,FIUME E.Aesthetic edits for character animation[C]//Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation.San Diego,California:ACM,2003. [58]COLEMAN P,BIBLIOWICZ J,SINGH K,et al.Staggered poses:a character motion representation for detail-preserving editing of pose and coordinated timing[C]//Eurographics/ACM SIGGRAPH Symposium on Computer Animation,SCA 2008.Dublin,Ireland,DBLP,2008:137-146. [59]GUAY M,GLEICHER M,CANI M P.Adding dynamics tosketch-based character animations[C]//Sketch-Based Interfaces and Modeling.2015:27-34. [60]IKEMOTO L,ARIKAN O,FORSYTH D.Generalizing motion edits with Gaussian processes[J].Acm Transactions on Graphi-cs,2009,28(1):1-12. [61]HSU E,PULLI K.Style translation for human motion[J].ACM Transactions on Graphics (TOG),2005,24(3):1082-1089. [62]GATYS L A,ECKER S,BETHGE M.A neural algorithm of artistic style[J].arXiv:1508.06576,2015. [63]GATYS L A,ECKER A S,BETHGE M.Image style transfer using convolutional neural networks[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Las Vegas,USA,2016:2414-2423. [64]BRAND M,HERTZMANN A.Style machines[C]//Procee-dings of the 27th annual conference on Computer graphics and interactive techniques.ACM Press/Addison-Wesley Publishing Co.,2000:183-192. [65]MIN J,LIU H,CHAI J.Synthesis and editing of personalizedstylistic human motion[C]//Symposium on Interactive 3d Graphics(Si3d 2010).Washington,DC,USA,DBLP,2010:39-46. [66]HOLDEN D,HABIBIE I,KUSAJIMA I,et al.Fast NeuralStyle Transfer for Motion Data[J].IEEE computer graphics and applications,2017,37(4):42-49. |
[1] | 周芳泉, 成卫青. 基于全局增强图神经网络的序列推荐 Sequence Recommendation Based on Global Enhanced Graph Neural Network 计算机科学, 2022, 49(9): 55-63. https://doi.org/10.11896/jsjkx.210700085 |
[2] | 周乐员, 张剑华, 袁甜甜, 陈胜勇. 多层注意力机制融合的序列到序列中国连续手语识别和翻译 Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion 计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026 |
[3] | 宁晗阳, 马苗, 杨波, 刘士昌. 密码学智能化研究进展与分析 Research Progress and Analysis on Intelligent Cryptology 计算机科学, 2022, 49(9): 288-296. https://doi.org/10.11896/jsjkx.220300053 |
[4] | 李宗民, 张玉鹏, 刘玉杰, 李华. 基于可变形图卷积的点云表征学习 Deformable Graph Convolutional Networks Based Point Cloud Representation Learning 计算机科学, 2022, 49(8): 273-278. https://doi.org/10.11896/jsjkx.210900023 |
[5] | 郝志荣, 陈龙, 黄嘉成. 面向文本分类的类别区分式通用对抗攻击方法 Class Discriminative Universal Adversarial Attack for Text Classification 计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077 |
[6] | 王润安, 邹兆年. 基于物理操作级模型的查询执行时间预测方法 Query Performance Prediction Based on Physical Operation-level Models 计算机科学, 2022, 49(8): 49-55. https://doi.org/10.11896/jsjkx.210700074 |
[7] | 陈泳全, 姜瑛. 基于卷积神经网络的APP用户行为分析方法 Analysis Method of APP User Behavior Based on Convolutional Neural Network 计算机科学, 2022, 49(8): 78-85. https://doi.org/10.11896/jsjkx.210700121 |
[8] | 朱承璋, 黄嘉儿, 肖亚龙, 王晗, 邹北骥. 基于注意力机制的医学影像深度哈希检索算法 Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism 计算机科学, 2022, 49(8): 113-119. https://doi.org/10.11896/jsjkx.210700153 |
[9] | 檀莹莹, 王俊丽, 张超波. 基于图卷积神经网络的文本分类方法研究综述 Review of Text Classification Methods Based on Graph Convolutional Network 计算机科学, 2022, 49(8): 205-216. https://doi.org/10.11896/jsjkx.210800064 |
[10] | 闫佳丹, 贾彩燕. 基于双图神经网络信息融合的文本分类方法 Text Classification Method Based on Information Fusion of Dual-graph Neural Network 计算机科学, 2022, 49(8): 230-236. https://doi.org/10.11896/jsjkx.210600042 |
[11] | 金方焱, 王秀利. 融合RACNN和BiLSTM的金融领域事件隐式因果关系抽取 Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM 计算机科学, 2022, 49(7): 179-186. https://doi.org/10.11896/jsjkx.210500190 |
[12] | 彭双, 伍江江, 陈浩, 杜春, 李军. 基于注意力神经网络的对地观测卫星星上自主任务规划方法 Satellite Onboard Observation Task Planning Based on Attention Neural Network 计算机科学, 2022, 49(7): 242-247. https://doi.org/10.11896/jsjkx.210500093 |
[13] | 费星瑞, 谢逸. 基于HMM-NN的用户点击流识别 Click Streams Recognition for Web Users Based on HMM-NN 计算机科学, 2022, 49(7): 340-349. https://doi.org/10.11896/jsjkx.210600127 |
[14] | 赵冬梅, 吴亚星, 张红斌. 基于IPSO-BiLSTM的网络安全态势预测 Network Security Situation Prediction Based on IPSO-BiLSTM 计算机科学, 2022, 49(7): 357-362. https://doi.org/10.11896/jsjkx.210900103 |
[15] | 齐秀秀, 王佳昊, 李文雄, 周帆. 基于概率元学习的矩阵补全预测融合算法 Fusion Algorithm for Matrix Completion Prediction Based on Probabilistic Meta-learning 计算机科学, 2022, 49(7): 18-24. https://doi.org/10.11896/jsjkx.210600126 |
|