计算机科学 ›› 2022, Vol. 49 ›› Issue (10): 214-223.doi: 10.11896/jsjkx.210900080
代福芸, 迟静, 任明国, 张琪东
DAI Fu-yun, CHI Jing, REN Ming-guo, ZHANG Qi-dong
摘要： 针对当前人脸图像合成中存在的模样和表情合成的多样性不足、表情的真实感不高、合成效率低的问题,提出了一种引入面部几何特征和属性标签约束的新型人脸合成网络模型。该模型在给定一张源人脸图片、一张目标人脸图片和属性(如发色、性别、年龄等)标签的情况下,能够生成一张具有源人脸表情、目标人脸身份特征以及指定属性的高真实感的人脸图像。模型包括两部分:人脸特征点生成器(Facial Landmark Generator,FLMG)和几何-属性感知生成器(Geometry and Attribute Aware Generator,GAAG)。FLMG利用人脸几何特征点编码表情信息,并将源人脸的表情信息迁移到目标人脸的特征点上;GAAG结合FLMG中生成的特征点、给定的属性标签和目标人脸图片,生成一张具有指定模样和表情的人脸图片。文中还引入了一个新的软截断三元感知损失函数用于约束GAAG,该函数可使生成的人脸更好地保持目标人脸的身份特征且更加真实自然,同时可使GAAG模型以更快的速度收敛。实验结果表明,所提方法可合成模样和表情多样化的人脸图像,且人脸的外貌真实、表情自然。另外,所提网络模型只需训练一次即可实现任意不同表情之间的迁移,效率较高。
中图分类号:
| [1]SUSSKIND J M,HINTON G E,MOVELLAN J R,et al.Gene-rating facial expressions with deep belief nets[M]//Affective Computing,Emotion Modelling,Synthesis and Recognition.2008:421-440. [2]YAN Y F,LU K,XUE J,et al.Face animation method based on deep learning and expression AU parameters [J].Journal of Computer Aided Design & Graphics,2019,31(11):1973-1980. [3]GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al.Generative adversarial nets[C]//Neural Information Processing Systems.2014:2672-2680. [4]ARJOVSKY M,CHINTALA S,BOTTOU L.Wasserstein GAN[J].arXiv:1701.07875,2017. [5]ISOLA P,ZHU J Y,ZHOU T H,et al.Image-to-image translation with conditional adversarial networks[C]//Computer Vison and Pattern Recognition.2017:5967-5976. [6]YI Z L,ZHANG H,TAN P,et al.DualGAN:unsupervised dual learning for image-to-image translation[C]//IEEE International Conference on Computer Vision.2017:2868-2876. [7]HUANG H B.Face image synthesis and analysis based on gene-rative model[D].Beijing:University of Chinese Academy of Sciences,2019. [8]CHEN F J,ZHU F,WU Q X,et al.A Survey of Generative Adversarial Networks and Its Application in Image Generation[J].Chinese Journal of Computers,2021,44(2):347-369. [9]PERARNAU G,VAN J,RADUCANU B,et al.Invertible conditional GANs for image editing[C]//NIPS Workshop on Adversarial Training.2016. [10]DING H,SRICHARAN K,CHELLAPPA R.ExprGAN:Facial expression editing with controllable expression intensity[C]//AAAI Conference on Artificial Intelligence.2018:6665-7655. [11]KARRAS T,LAINE S,AILA T.A Style-Based Generator Architecture for Generative Adversarial Networks[C]//Confe-rence on Computer Vision and Pattern Recognition.2019:4401-4410. [12]TEWARI A,ELGHARIB M,BHARAJ G.StyleRig:RiggingStyleGAN for 3D Control over Portrait Images[C]//Confe-rence on Computer Vision and Pattern Recognition.2020:6141-6150. [13]SONG L X,LU Z H,HE R,et al.Geometry Guided Adversarial Facial Expression Synthesis[C]//ACM International Confe-rence on Multimedia.2018:627-635. [14]QIAO F C,YAO N M,JIAO Z R,et al.Emotional facial expression transfer from a single image via generative adversarial nets[J].Computer Animation and Virtual Worlds,2018,29(3/4):1/10-10/10. [15]OLSZEWSKI Z,LI Z M,YANG C,et al.Realistic Dynamic Facial Textures from a Single Image Using GANs[C]//IEEE International Conference on Computer Vision.2017:5429-5438. [16]ZHU J Y,PARK T,ISOLA P,et al.Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks[C]//IEEE International Conference on Computer Vision.2017:2232-2969. [17]CHOI Y J,CHOI M J,KIM M Y,et al.StarGAN:Unified Gene-rative Adversarial Networks for Multi-Domain Image-to-Image Translation[C]//IEEE Conference on Computer Vision and Pattern Recognition.2018:8789-8797. [18]ZHANG J N,ZENG X F,WANG M M,et al.FReeNet:Multi-Identity Face Reenactment[C]//Computer Vison and Pattern Recognition.2020:5326-5335. [19]WANG X P,WANG Y H,LI W X.U-Net Conditional GANs for Photo-Realistic and Identity-Preserving Facial Expression Synthesis[J].ACM Transactions on Multimedia Computing,Communications,and Applications,2019,15(3s):1-23. [20]SCHROFF F,KALENICHENKO D,PHILBIN J.FaceNet:AUnified Embedding for Face Recognition and Clustering[C]//Computer Vison and Pattern Recognition.2015:815-823. [21]JUSTIN J,ALEXANDRE A,LI F F.Perceptual Losses forReal-Time Style Transfer and Super-Resolution[C]//European Conference on Computer Vision.2016:694-711. [22]GUO X J,LI S Y,ZHANG J W,et al.PFLD:A Practical Facial Landmark Detector[J].arXiv:1902.10859,2019. [23]ODENA A,OLAH C,SHLENS J.Conditional Image Synthesis with Auxiliary Classifier GANs[C]//The International Confe-rence on Machine Learning.2017:2642-2651. [24]SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-Scale Image Recognition[C]//International Conference on Learning Representations.2015. [25]LANGNER O,DOTSCH R,BIJLSTRA G,et al.Presentation and validation of the Radboud Faces Database [J].Cognition and Emotion,2010,24(8):1377-1388. [26]Karolinska Institute,Stockholm,Sweden.The Karolinska Di-rected Emotional Faces[EB/OL].https://www.kdef.se/index.html. [27]DIEDERIK P,KINGMA D,JIMMY B.Adam:A Method for Stochastic Optimization[C]//International Conference on Learning Representations.2015. [28]AMAZON.Amazon Mechanical Turk(AMT)[EB/OL].https://www.mturk.com. |
| [1] | 徐国宁, 陈奕芃, 陈一鸣, 陈晋音, 温浩. 基于约束优化生成式对抗网络的数据去偏方法 Data Debiasing Method Based on Constrained Optimized Generative Adversarial Networks 计算机科学, 2022, 49(6A): 184-190. https://doi.org/10.11896/jsjkx.210400234 |
| [2] | 唐雨潇, 王斌君. 基于深度生成模型的人脸编辑研究进展 Research Progress of Face Editing Based on Deep Generative Model 计算机科学, 2022, 49(2): 51-61. https://doi.org/10.11896/jsjkx.210400108 |
| [3] | 邢红杰, 郝忠. 基于全局和局部判别对抗自编码器的异常检测方法 Novelty Detection Method Based on Global and Local Discriminative Adversarial Autoencoder 计算机科学, 2021, 48(6): 202-209. https://doi.org/10.11896/jsjkx.200400083 |
| [4] | 刘海波,武天博,沈晶,史长亭. 基于GAN-LSTM的APT攻击检测 Advanced Persistent Threat Detection Based on Generative Adversarial Networks and Long Short-term Memory 计算机科学, 2020, 47(1): 281-286. https://doi.org/10.11896/jsjkx.181102103 |
| [5] | 刘剑, 金泽群. 基于深度学习的人脸表情迁移方法 Facial Expression Transfer Method Based on Deep Learning 计算机科学, 2019, 46(6A): 250-253. |
|
||
首页