计算机科学

计算机科学 ›› 2019, Vol. 46 ›› Issue (9): 22-27.doi: 10.11896/j.issn.1002-137X.2019.09.003

• 综述 • 上一篇    下一篇

基于神经网络的角色运动合成研究进展

王鑫1,2, 孟浩浩1,2, 姜小涛1,2, 陈胜勇1,3, 孙凌云4,5   

  1. (浙江工业大学计算机科学与技术学院 杭州310023)1;
    (浙江省可视媒体智能处理技术研究重点实验室 杭州310023)2;
    (天津理工大学计算机科学与工程学院 天津300384)3;
    (浙江大学现代工业设计研究院 杭州310058)4;
    (浙江大学国际设计研究院 杭州310058)5
  • 收稿日期:2018-11-14 出版日期:2019-09-15 发布日期:2019-09-02
  • 通讯作者: 王 鑫(1984-),男,博士,副教授,CCF会员,主要研究方向为计算机视觉、计算机图形学,E-mail:xinw@zjut.edu.cn
  • 作者简介:孟浩浩(1993-),男,硕士生,主要研究方向为计算机图形学;姜小涛(1992-),男,硕士生,主要研究方向为计算机图形学;陈胜勇(1973-),男,博士,教授,博士生导师,主要研究方向为计算机视觉、图像分析与处理、机器人智能技术;孙凌云(1981-),男,博士,教授,博士生导师,主要研究方向为人工智能、设计智能、信息与交互设计。
  • 基金资助:
    国家自然科学基金(61303142,61004116,61672451,U1509207),浙江省自然科学基金(Y1110882)

Survey on Character Motion Synthesis Based on Neural Network

WANG Xin1,2, MENG Hao-hao1,2, JIANG Xiao-tao1,2, CHEN Sheng-yong1,3, SUN Ling-yun4,5   

  1. (College of Computer Science and Technology,Zhejiang University of Technology,Hangzhou 310023,China)1;
    (Key Laboratory of Visual Media Intelligent Process Technology of Zhejiang Province,Hangzhou 310023,China)2;
    (College of Computer Science and Engineering,Tianjin University of Technology,Tianjin 300384,China)3;
    (Modern Industrial Design Institute,Zhejiang University,Hangzhou 310058,China)4;
    (International Design Institute,Zhejiang University,Hangzhou 310058,China)5
  • Received:2018-11-14 Online:2019-09-15 Published:2019-09-02

PDF (PC)

1509

摘要： 在角色运动数据集上,运用神经网络技术进行运动合成是当前计算机图形学领域中的一项重要研究。该研究旨在通过神经网络技术生成自然、逼真度较高的角色运动。在对相关研究工作进行分析和总结的基础上,对运动模型的构建、运动交互和运动风格化等领域的研究进展进行了介绍;详细阐述了基于运动捕获数据,利用数据驱动技术、交互式控制方法和ERD,CAE,MAR等网络模型,动态地对角色进行运动建模、运动合成、交互式运动控制,同时为了合成更高质量的角色运动,对运动动画进行风格化等处理;以神经网络技术为着眼点,串联角色运动合成中的各个环节,并结合实际应用,针对当前研究工作面临的难点提出一些可继续深入探索的问题。

关键词: 风格编辑, 交互式角色控制, 角色运动合成, 神经网络, 数据驱动

Abstract: The application of neural network technology to character motion synthesis on human motion data sets is an important research content in the field of computer graphics.This study aims to generate naturally realistic character motion using neural networks through date-driven technology.Based on the analysis and summary of related research work,this paper introduced the research progress in the fields of motion model construction,motion interaction and motion stylization and so on.Based on the motion capture data,by using data-driven technology,interactive control methods and network models such as ERD,CAE and MAR,the character was dynamically modeled,synthesized and controlled by interactive motion,and in order to generate higher quality character motions,motion animation and other content were stylized.In this paper,taking neural network technology as the focal point,various study works of the character motion synthesis were connected.Combined with the practical applications and difficulties faced in the current research work,this paper suggested some problems that can be further studied.

Key words: Character motion synthesis, Data driven, Interactive character control, Neural network, Style editting

中图分类号: 

  • TP311
[1]GUO S,SOUTHERN R,CHANG J,et al.Adaptive motion synthesis for virtual characters:a survey[J].The Visual Computer,2015,31(5):497-512.
[2]WANG X,CHEN Q,WANG W.3D Human Motion Editing and Synthesis:A Survey[J].Computational and Mathematical Methods in Medicine,2014,2014(3):104535.
[3]LIU G D,PANG Z G,CHENG X,et al.A Survey on Machine Learning in the Synthesis of Human Motions[J].Journal of Computer-Aided Design and Computer Graphics,2010,22(9):1619-1627.(in Chinese)刘更代,潘志庚,程熙,等.人体运动合成中的机器学习技术综述[J].计算机辅助设计与图形学学报,2010,22(9):1619-1627.
[4]KRIZHEVSKY A,SUTSKEVER I,HINTON G E.Imagenetclassification with deep convolutional neural networks[J].Advances in Neural Information Processing Systems,2012,25(2):1097-1105.
[5]SHAZEER N,MIRHOSEINI A,MAZIARZ K,et al.Outra-geously large neural networks:The sparsely-gated mixture-of-experts layer[J].arXiv:1701.06538,2017.
[6]PENG X B,ABBEEL P,LEVINE S,et al.DeepMimic:Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills[J].arXiv:1804.02717,2018.
[7]GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al.Generative adversarial nets[C]//International Conference on Neural Information Processing Systems.MIT Press,Montreal,Canada,2014:2672-2680.
[8]GENG J,SHAO T,ZHENG Y,et al.Warp-guided GANs for single-photo facial animation[C]//SIGGRAPH Asia 2018 Technical Papers.ACM,Tokyo,Japan,2018:231.
[9]KELLY T,GUERRERO P,STEED A,et al.FrankenGAN:Guided Detail Synthesis for Building Mass-Models Using Style-Synchonized GANs[J].arXiv:1806.07179,2018.
[10]BENGIO E,BACON P L,PINEAU J,et al.Conditional computation in neural networks for faster models[J].arXiv:1511.06297,2015.
[11]FU J,CO-REYES J,LEVINE S.Ex2:Exploration with exem-plar models for deep reinforcement learning[C]//Advances in Neural Information Processing Systems.Long Beach,USA,2017:2577-2587.
[12]ZHAO J J,WEI Y,XIA S H,et al.Survey of Physics-BasedCharacter Animation [J].Journal of Computer Research and Development,2015,52(12):2866-2878.(in Chinese)赵建军,魏毅,夏时洪,等.基于物理的角色动画合成方法综述[J].计算机研究与发展,2015,52(12):2866-2878.
[13]ALLEN B F,FALOUTSOS P.Evolved Controllers for Simulated Locomotion[C]//International Workshop on Motion in Games.Springer-Verlag,2009:219-230.
[14]TAN J,GU Y,LIU C K,et al.Learning bicycle stunts[J].Acm Transactions on Graphics,2014,33(4):50.
[15]MORDATCH I,LOWREY K,ANDREW G,et al.Interactive control of diverse complex characters with neural networks[C]//Advances in Neural Information Processing Systems.Long Beach,USA,2015:3132-3140.
[16]LEVINE S,KOLTUN V.Learning Complex Neural Network Policies with Trajectory Optimization[C]//International Conference on Machine Learning.Beijing,China,2014:829-837.
[17]DU Y,WANG W,WANG L.Hierarchical recurrent neural network for skeleton based action recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Boston,USA,2015:1110-1118.
[18]HOLDEN D,SAITO J,KOMURA T,et al.Learning motionmanifolds with convolutional autoencoders[C]//SIGGRAPH Asia 2015 Technical Briefs.Kobe,Japan,2015:18.
[19]HOLDEN D,KOMURA T,SAITO J.Phase-functioned neural networks for character control[J].Acm Transactions on Gra-phics,2017,36(4):1-13.
[20]HOLDEN D,SAITO J,KOMURA T.A deep learning framework for character motion synthesis and editing[J].Acm Transactions on Graphics,2016,35(4):1-11.
[21]XIA S H,WEI Y,WANG Z Q.A Survey of Physics-Based Human Motion Simulation[J].Journal of Computer Research and Development,2010,47(8):1354-1361.(in Chinese)夏时洪,魏毅,王兆其.人体运动仿真综述[J].计算机研究与发展,2010,47(8):1354-1361.
[22]MITTELMAN R,KUIPERS B,SAVARESE S,et al.Structuredrecurrent temporal restricted boltzmann machines[C]//International Conference on Machine Learning.JMLR.org,2014:II-1647.
[23]FRAGKIADAKI K,LEVINE S,FELSEN P,et al.Recurrentnetwork models for human dynamics[C]//Proceedings of the IEEE International Conference on Computer Vision.San Diego,USA,2015:4346-4354.
[24]SAITO S,HU L,MA C,et al.3D hair synthesis using volumetric variational autoencoders[C]//SIGGRAPH Asia 2018 Technical Papers.Tokyo,Japan,2018:208.
[25]CHU M,THUEREY N.Data-driven synthesis of smoke flows with CNN-based feature descriptors[J].ACM Transactions on Graphics (TOG),2017,36(4):69.
[26]XIA S,WANG C,CHAI J,et al.Realtime style transfer for unlabeled heterogeneous human motion[J].ACM Transactions on Graphics,2015,34(4):1-10.
[27]LI Z,ZHOU Y,XIAO S,et al.Auto-conditioned recurrent networks for extended complex human motion synthesis[J].arXiv:1707.05363,2017.
[28]BARSOUM E,KENDER J,LIU Z.HP-GAN:Probabilistic 3Dhuman motion prediction via GAN[J].arXiv:1711.09561,2017.
[29]RAJESWARAN A,KUMAR V,GUPTA A,et al.Learningcomplex dexterous manipulation with deep reinforcementlear-ning and demonstrations[J].arXiv:1709.10087,2017.
[30]MA P,TIAN Y,PAN Z,et al.Fluid directed rigid body control using deep reinforcement learning[J].ACM Transactions on Graphics (TOG),2018,37(4):96.
[31]HUANG T C,HUANG Y J,LIN W C.Real-time horse gaitsynthesis[J].Computer Animation and Virtual Worlds,2013,24(2):87-95.
[32]ZHANG H,STARKE S,KOMURA T,et al.Mode-adaptiveneural networks for quadruped motion control[J].ACM Tran-sactions on Graphics (TOG),2018,37(4):145.
[33]LEE K,LEE S,LEE J.Interactive character animation by lear-ning multi-objective control[C]//SIGGRAPH Asia 2018 Technical Papers.Tokyo,Japan,2018:180.
[34]HO J,ERMON S.Generative adversarial imitation learning[C]//Advances in Neural Information Processing Systems.Barcelona,Spain,2016:4565-4573.
[35]LIU L,HODGINS J.Learning to schedule control fragments for physics-based characters using deep q-learning[J].ACM Transactions on Graphics (TOG),2017,36(3):29.
[36]LAU M,KUFFNER J J.Behavior planning for character animation[C]//ACM Siggraph/eurographics Symposium on ComputerAnimation.Los Angeles,USA,2005:271-280.
[37]SAFONOVA A,HODGINS J K.Construction and optimalsearch of interpolated motion graphs[J].ACM Transactions on Graphics,2007,26(3):106.
[38]LIU L,YIN K K,VAN DE PANNE M,et al.Sampling-based contact-rich motion control[C]//ACM Transactions on Gra-phics (TOG).New York,USA,2010,29(4):128.
[39]MIN J,CHAI J.Motion graphs++:a compact generative model for semantic motion analysis and synthesis[J].ACM Transactions on Graphics (TOG),2012,31(6):153.
[40]HWANG J,KIM J,SUH I H,et al.Real-time Locomotion Controller using an Inverted-Pendulum-based Abstract Model[J].Computer Graphics Forum,2018,37(2):287-296.
[41]RAJESWARAN A,GHOTRA S,RAVINDRAN B,et al.Epopt:Learning robust neural network policies using model ensembles[J].arXiv:1610.01283,2016.
[42]SU W,DU D,YANG X,et al.Interactive sketch-based normal map generation with deep neural networks[J].Proceedings of the ACM on Computer Graphics and Interactive Techniques,2018,1(1):22.
[43]LEVINE S,WANG J M,HARAUX A,et al.Continuous character control with low-dimensional embeddings[J].ACM Tran-sactions on Graphics (TOG),2012,31(4):28.
[44]CLEGG A,YU W,TAN J,et al.Learning to dress:synthesizing human dressing motion via deep reinforcement learning[C]//SIGGRAPH Asia 2018 Technical Papers.Tokyo,Japan,2018:179.
[45]PENG X B,BERSETH G,VAN DE PANNE M.Terrain-adaptive locomotion skills using deep reinforcement learning[J].ACM Transactions on Graphics (TOG),2016,35(4):81.
[46]LIU L,HODGINS J.Learning basketball dribbling skills using trajectory optimization and deep reinforcement learning[J].ACM Transactions on Graphics (TOG),2018,37(4):142.
[47]NAKADA M,CHEN H,TERZOPOULOS D.Learning Biomi-metic Perception for Human Sensorimotor Control[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops.Salt Lake City,USA,2018:1917-1922.
[48]NAKADA M,ZHOU T,CHEN H,et al.Deep learning of biomimetic sensorimotor control for biomechanical human animation[J].ACM Transactions on Graphics (TOG),2018,37(4):56.
[49]NAIR A,MCGREW B,ANDRYCHOWICZ M,et al.Overco-ming exploration in reinforcement learning with demonstrations[C]//2018 IEEE International Conference on Robotics and Automation (ICRA).IEEE,Brisbane,Australis,2018:6292-6299.
[50]GUPTA A,SATKIN S,EFROS A A,et al.From 3d scene geometry to human workspace[C]//2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE,2011:1961-1968.
[51]KIM V G,CHAUDHURI S,GUIBAS L,et al.Shape2pose:Human-centric shape analysis[J].ACM Transactions on Graphics (TOG),2014,33(4):120.
[52]SAVVA M,CHANG A X,Hanrahan P,et al.PiGraphs:learning interaction snapshots from observations[J].ACM Transactions on Graphics (TOG),2016,35(4):139.
[53]KAPADIA M,XU X H,NITTI M,et al.Precision:Precompu-ting environment semantics for contact-rich character animation[C]//Proceedings of the 20th ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games.ACM,2016:29-37.
[54]CHANG X,HOSPEDALES T M,XIANG T.Multi-level factorisation net for person re-identification[C]//CVPR.Salt Lake City,USA,2018:2.
[55]KULKARNI T D,WHITNEY W F,KOHLI P,et al.Deep convolutional inverse graphics network[C]//Advances in Neural Information Processing Systems.Long Beach,USA,2015:2539-2547.
[56]CHOI B,LEWIS J P,SEOL Y,et al.SketchiMo:sketch-basedmotion editing for articulated characters[J].ACM Transactions on Graphics (TOG),2016,35(4):146.
[57]NEFF M,FIUME E.Aesthetic edits for character animation[C]//Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation.San Diego,California:ACM,2003.
[58]COLEMAN P,BIBLIOWICZ J,SINGH K,et al.Staggered poses:a character motion representation for detail-preserving editing of pose and coordinated timing[C]//Eurographics/ACM SIGGRAPH Symposium on Computer Animation,SCA 2008.Dublin,Ireland,DBLP,2008:137-146.
[59]GUAY M,GLEICHER M,CANI M P.Adding dynamics tosketch-based character animations[C]//Sketch-Based Interfaces and Modeling.2015:27-34.
[60]IKEMOTO L,ARIKAN O,FORSYTH D.Generalizing motion edits with Gaussian processes[J].Acm Transactions on Graphi-cs,2009,28(1):1-12.
[61]HSU E,PULLI K.Style translation for human motion[J].ACM Transactions on Graphics (TOG),2005,24(3):1082-1089.
[62]GATYS L A,ECKER S,BETHGE M.A neural algorithm of artistic style[J].arXiv:1508.06576,2015.
[63]GATYS L A,ECKER A S,BETHGE M.Image style transfer using convolutional neural networks[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Las Vegas,USA,2016:2414-2423.
[64]BRAND M,HERTZMANN A.Style machines[C]//Procee-dings of the 27th annual conference on Computer graphics and interactive techniques.ACM Press/Addison-Wesley Publishing Co.,2000:183-192.
[65]MIN J,LIU H,CHAI J.Synthesis and editing of personalizedstylistic human motion[C]//Symposium on Interactive 3d Graphics(Si3d 2010).Washington,DC,USA,DBLP,2010:39-46.
[66]HOLDEN D,HABIBIE I,KUSAJIMA I,et al.Fast NeuralStyle Transfer for Motion Data[J].IEEE computer graphics and applications,2017,37(4):42-49.
[1] 周芳泉, 成卫青.
基于全局增强图神经网络的序列推荐
Sequence Recommendation Based on Global Enhanced Graph Neural Network
计算机科学, 2022, 49(9): 55-63. https://doi.org/10.11896/jsjkx.210700085
[2] 周乐员, 张剑华, 袁甜甜, 陈胜勇.
多层注意力机制融合的序列到序列中国连续手语识别和翻译
Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion
计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026
[3] 宁晗阳, 马苗, 杨波, 刘士昌.
密码学智能化研究进展与分析
Research Progress and Analysis on Intelligent Cryptology
计算机科学, 2022, 49(9): 288-296. https://doi.org/10.11896/jsjkx.220300053
[4] 李宗民, 张玉鹏, 刘玉杰, 李华.
基于可变形图卷积的点云表征学习
Deformable Graph Convolutional Networks Based Point Cloud Representation Learning
计算机科学, 2022, 49(8): 273-278. https://doi.org/10.11896/jsjkx.210900023
[5] 郝志荣, 陈龙, 黄嘉成.
面向文本分类的类别区分式通用对抗攻击方法
Class Discriminative Universal Adversarial Attack for Text Classification
计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[6] 王润安, 邹兆年.
基于物理操作级模型的查询执行时间预测方法
Query Performance Prediction Based on Physical Operation-level Models
计算机科学, 2022, 49(8): 49-55. https://doi.org/10.11896/jsjkx.210700074
[7] 陈泳全, 姜瑛.
基于卷积神经网络的APP用户行为分析方法
Analysis Method of APP User Behavior Based on Convolutional Neural Network
计算机科学, 2022, 49(8): 78-85. https://doi.org/10.11896/jsjkx.210700121
[8] 朱承璋, 黄嘉儿, 肖亚龙, 王晗, 邹北骥.
基于注意力机制的医学影像深度哈希检索算法
Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism
计算机科学, 2022, 49(8): 113-119. https://doi.org/10.11896/jsjkx.210700153
[9] 檀莹莹, 王俊丽, 张超波.
基于图卷积神经网络的文本分类方法研究综述
Review of Text Classification Methods Based on Graph Convolutional Network
计算机科学, 2022, 49(8): 205-216. https://doi.org/10.11896/jsjkx.210800064
[10] 闫佳丹, 贾彩燕.
基于双图神经网络信息融合的文本分类方法
Text Classification Method Based on Information Fusion of Dual-graph Neural Network
计算机科学, 2022, 49(8): 230-236. https://doi.org/10.11896/jsjkx.210600042
[11] 金方焱, 王秀利.
融合RACNN和BiLSTM的金融领域事件隐式因果关系抽取
Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM
计算机科学, 2022, 49(7): 179-186. https://doi.org/10.11896/jsjkx.210500190
[12] 彭双, 伍江江, 陈浩, 杜春, 李军.
基于注意力神经网络的对地观测卫星星上自主任务规划方法
Satellite Onboard Observation Task Planning Based on Attention Neural Network
计算机科学, 2022, 49(7): 242-247. https://doi.org/10.11896/jsjkx.210500093
[13] 费星瑞, 谢逸.
基于HMM-NN的用户点击流识别
Click Streams Recognition for Web Users Based on HMM-NN
计算机科学, 2022, 49(7): 340-349. https://doi.org/10.11896/jsjkx.210600127
[14] 赵冬梅, 吴亚星, 张红斌.
基于IPSO-BiLSTM的网络安全态势预测
Network Security Situation Prediction Based on IPSO-BiLSTM
计算机科学, 2022, 49(7): 357-362. https://doi.org/10.11896/jsjkx.210900103
[15] 齐秀秀, 王佳昊, 李文雄, 周帆.
基于概率元学习的矩阵补全预测融合算法
Fusion Algorithm for Matrix Completion Prediction Based on Probabilistic Meta-learning
计算机科学, 2022, 49(7): 18-24. https://doi.org/10.11896/jsjkx.210600126
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发