基于深度学习的人脸表情迁移方法

摘要/Abstract

摘要： 针对人脸表情迁移生成图像质量不高、训练过程较长且生成速度较慢的问题,文中提出了一种基于生成式对抗网络的人脸表情迁移方法,使表情迁移更加快速和自然。首先,利用卷积神经网络进行人脸特征提取,并将图像从高维空间映射到浅层空间,在浅层空间中利用生成式对抗网络模型对人脸表情特征进行判别;然后,通过最近邻上采样层和卷积层组合结构将图像从浅层空间映射到高维空间,并在此过程中通过加入表情标签特征图对人脸表情进行改变。与Fader Networks相比,所提方法的网络模型参数量减少43.7%,训练时间缩短了36%。实验结果表明,所提方法有效地提高了人脸表情迁移生成图像的速度和质量。

关键词: 计算机视觉, 人脸表情迁移, 深度学习, 生成式对抗网络

Abstract: In order to solve the problems of low image quality,long training process and slow generation speed of face expression transfer,this paper proposed a facial expression transfermethod based on generative adversarial network to make expression transfer faster and more natural.Firstly,the facial features are extracted by using convolutional neural network,and the images are mapped from high-dimensional space to shallow space.In the shallow space,the facial expression features are discriminated by using the Generative Adversarial Networks.Then the nearest neighbors up-sampling and convolutional neural networks are used to mapthe image from the shallow space to the high-dimensional space,and in this process,the face expression is changed by adding the facial expression feature maps into neural networks.Compared with Fader Networks,the network model parameter amount of the proposed method is reduced by 43.7% and training time is reduced by 36%.The experimental results show that the proposed method can effectively improve the quality and the speed of generated images.

Key words: Computer vision, Deep learning, Face expression transfer, Generative adversarial networks

中图分类号:

TP183

刘剑, 金泽群. 基于深度学习的人脸表情迁移方法[J]. 计算机科学, 2019, 46(6A): 250-253. https://doi.org/

LIU Jian, JIN Ze-qun. Facial Expression Transfer Method Based on Deep Learning[J]. Computer Science, 2019, 46(6A): 250-253. https://doi.org/

参考文献

[1]IOFFE S,SZEGEDY C:Batch Normalization:Accelerating Deep Network Training by Reducing Internal Covariate Shift[J].ar-Xiv:1502.03167v3,2015.
[2]ISOLA P,ZHU J Y,ZHOU T H,et al.Image-to-image translation with conditional adversarial networks[J].arXiv:1611.07004,2016.
[3]王娅,侯进,王献.基于顶点权重的网格简化在虚拟人脸中的应用[J].计算机仿真,2014,31(2):329-334.
[4]雷腾.虚拟人眼的运动与表情合成的研究[D].成都:西南交通大学,2014.
[5]李俊龙,章登义,黄珺.Kinect驱动的人脸动画合成技术研究[J].计算机工程,2015,41(3):237-241.
[6]ZHANG H,XU T,LI H,et al.Stackgan:Text to photo-realistic image synthesis with stacked generative adversarial networks[J].arXiv:1612.03242,2016.
[7]ZHANG Z,SONG Y,QI H.Age progression/regression by conditional adversarial autoencoder[C]∥The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE,2017.
[8]ZHAO J,MATHIEU M,LECUN Y.Energy-based generative adversarial network[C]∥5th International Conference on Learning Representations (ICLR).2017.
[9]LAMPLE G,ZEGHIDOUR N.Fader Networks:Manipulating Images by Sliding Attributes[J].arXiv:1706.00409v2,2017.
[10]HINTON G,KRIZHEVSKY A,WANG S.Transforming auto-encoders[C]∥Artificial Neural Networks and Machine Learning(ICANN 2011).2011:44-51.
[11]PERARNAU G,VAN DE WEIJER J,RADUCANU B,et al.Invertible conditional gans for image editing[J].arXiv:1611.06355,2016.
[12]RATLIFF L J,BURDEN S A,SASTRY S S.Characterization and computation of local Nash equilibria in continuous games[C]∥Proceedings of the 51st Annual Allerton Conference on Communication,Control,and Computing (Allerton).Monticello,IL,USA,IEEE,2013:917-924.
[13]ARJOVSKY M,OTTOU L.Towards principled methods for training generative adversarial networks[C]∥ICLR.2017.
[14]SHEN W,LIU R.Learning residual images for face attribute manipulation[C]∥The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).2017.
[15]ANTIPOV G,BACCOUCHE M,DUGELAY J L.Face aging with conditional generative adversarial networks[J].arXiv:1702.01983,2017.
[16]LIU Z W,LUO P,WANG X G,et al.Deep learning face attri-butes in the wild[C]∥Proceedings of International Conference on Computer Vision (ICCV).2015.
[17]GLOROT X,BENGIO Y.Understanding the difficulty of training deep feedforward neural networks[C]∥AISTATS.2010.
[18]KINGMA D,BA J.Adam:A method for stochastic optimization[J].arXiv:1412.6980,2014.

相关文章 15

[1]	饶志双, 贾真, 张凡, 李天瑞. 基于Key-Value关联记忆网络的知识图谱问答方法 Key-Value Relational Memory Networks for Question Answering over Knowledge Graph 计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[2]	汤凌韬, 王迪, 张鲁飞, 刘盛云. 基于安全多方计算和差分隐私的联邦学习方案 Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy 计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[3]	徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺. 时序知识图谱表示学习 Temporal Knowledge Graph Representation Learning 计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[4]	王剑, 彭雨琦, 赵宇斐, 杨健. 基于深度学习的社交网络舆情信息抽取方法综述 Survey of Social Network Public Opinion Information Extraction Based on Deep Learning 计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[5]	郝志荣, 陈龙, 黄嘉成. 面向文本分类的类别区分式通用对抗攻击方法 Class Discriminative Universal Adversarial Attack for Text Classification 计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[6]	姜梦函, 李邵梅, 郑洪浩, 张建朋. 基于改进位置编码的谣言检测模型 Rumor Detection Model Based on Improved Position Embedding 计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[7]	孙奇, 吉根林, 张杰. 基于非局部注意力生成对抗网络的视频异常事件检测方法 Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection 计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[8]	胡艳羽, 赵龙, 董祥军. 一种用于癌症分类的两阶段深度特征选择提取算法 Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification 计算机科学, 2022, 49(7): 73-78. https://doi.org/10.11896/jsjkx.210500092
[9]	程成, 降爱莲. 基于多路径特征提取的实时语义分割方法 Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction 计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[10]	侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木. 中文预训练模型研究进展 Advances in Chinese Pre-training Models 计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[11]	周慧, 施皓晨, 屠要峰, 黄圣君. 基于主动采样的深度鲁棒神经网络学习 Robust Deep Neural Network Learning Based on Active Sampling 计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
[12]	苏丹宁, 曹桂涛, 王燕楠, 王宏, 任赫. 小样本雷达辐射源识别的深度学习方法综述 Survey of Deep Learning for Radar Emitter Identification Based on Small Sample 计算机科学, 2022, 49(7): 226-235. https://doi.org/10.11896/jsjkx.210600138
[13]	刘伟业, 鲁慧民, 李玉鹏, 马宁. 指静脉识别技术研究综述 Survey on Finger Vein Recognition Research 计算机科学, 2022, 49(6A): 1-11. https://doi.org/10.11896/jsjkx.210400056
[14]	孙福权, 崔志清, 邹彭, 张琨. 基于多尺度特征的脑肿瘤分割算法 Brain Tumor Segmentation Algorithm Based on Multi-scale Features 计算机科学, 2022, 49(6A): 12-16. https://doi.org/10.11896/jsjkx.210700217
[15]	康雁, 徐玉龙, 寇勇奇, 谢思宇, 杨学昆, 李浩. 基于Transformer和LSTM的药物相互作用预测 Drug-Drug Interaction Prediction Based on Transformer and LSTM 计算机科学, 2022, 49(6A): 17-21. https://doi.org/10.11896/jsjkx.210400150

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed