Computer Science ›› 2021, Vol. 48 ›› Issue (1): 182-189.doi: 10.11896/jsjkx.191100092

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Anime Character Portrait Generation Algorithm Based on Improved Generative Adversarial Networks

ZHANG Yang, MA Xiao-hu   

  1. School of Computer Science & Technology,Soochow University,Suzhou,Jiangsu 215000,China
  • Received:2019-11-12 Revised:2020-03-20 Online:2021-01-15 Published:2021-01-15
  • About author:ZHANG Yang,born in 1996,master candidate,is a student member of China Computer Federation.His main research interests include generative adversarial networks and image processing.
    MA Xiao-hu,born in 1964,professor,master supervisor,is a advanced member of China Computer Federation.His main research interests include machine learning and image processing.
  • Supported by:
    Natural Science Foundation of Jiangsu,China(BK20141195) and Priority Academic Program Development of Jiangsu Higher Education Institutions.

Abstract: In order to solve the problems of poor diversity,generation by class and detail control in existed method,we present an improved model named LMV-ACGAN.It is based on ACGAN and involved with mutual information and multiscale discrimination.Our model includes a feature combined generator,a multiscale discriminator and three fully connected nets for real-fake judging,classifying and latent label restoration.As a semi-supervised generative model,except class label,we also use a group of continuous latent label to enhance the constraint of the generator.Moreover,in our algorithms,pooling layers in VGG-NET are replaced by stride convolutions.Then the discriminator uses the multiscale information of the image to feature fusion.Finally,we improve the tail-end structure of the model and the rules of parameters update so as to reduce the influence between classification,real-fake judgement and latent label restoration as far as possible.Our experiment shows that the proposed method effectively solve the problem of mode collapse on our dataset,meanwhile compared with origin ACGAN,our method increases the success rate and accuracy of generating specified class image.For the image which is generated poorly or classified incorrectly by ACGAN,our method can achieve the goal.In addition,our model enable people to modify the continuous latent label to realize image editing such as changing the face orientation.

Key words: ACGAN, Generative adversarial networks, Image edit, Image generation, Multi-scale discriminator

CLC Number: 

  • TP391
[1] GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al.Generative adversarial nets[C]//Conference on Neural Information Processing Systems.MIT Press,2014:2672-2680.
[2] KINGMA D P,WELLING M.Auto-encoding variational bayes[C]//International Conference on Learning Representations.ICLR,2014.
[3] RADFORD A,METZ L,CHINTALA S.Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks[J].arXiv:1511.06434,2016.
[4] ARJOVSKY M,CHINTALA S,BOTTOU L.Wasserstein ge-nerative adversarial networks [C]//International Conference on Machine Learning.ACM,2017:298-321.
[5] GULRAJANI I,AHMED F,ARJOVSKY M,et al.Improved training of wasserstein GANs [C]//Conference on Neural Information Processing Systems.MIT Press,2017:5768-5778.
[6] MIRZA M,Osindero S.Conditional Generative Adversarial Nets[J].arXiv:1411.1784,2014.
[7] ODENA A,OLAH C,SHLENS J.Conditional image synthesis with auxiliary classifier gans [C]//International Conference on Machine Learning.ACM,2017:4043-4055.
[8] CHEN X,DUAN Y,HOUTHOOFT R,et al.InfoGAN:Interpretable representation learning by information maximizing generative adversarial nets [C]//Conference on Neural Information Processing Systems.MIT Press,2016:2180-2188.
[9] KARNEWAR A,WANG O,IYENGAR R S.MSG-GAN:Multi-Scale Gradient GAN for Stable Image Synthesis [J].arXiv:1903.06048,2019:9.
[10] KARRAS T,AILA T,LAINE S,et al.Progressive growing of GANs for improved quality,stability,and variation[C]//International Conference on Learning Representations.ICLR,2018.
[11] YONGYI L,YU-WING T,CHI-KEUNG T.Attribute-GuidedFace Generation Using Conditional CycleGAN[C]//Computer Vision.15th European Conference(ECCV).Springer,2018:293-308.
[12] CHEN Y,LAI Y,LIU Y.Cartoon GAN:Generative Adversarial Networks for Photo Cartoonization [C]//Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2018:9465-9474.
[13] LIU Y,QIN Z,WAN T,et al.Auto-painter:Cartoon image generation from sketch by using conditional Wasserstein generative adversarial networks[J].Neurocomputing,2018,311:78-87.
[14] ZHANG L,JI Y,LIN X,et al.Style transfer for anime sketches with enhanced residual u-net and auxiliary classifier GAN [C]//Asian Conference on Pattern Recognition(ACPR).IEEE,2017:512-517.
[15] CI Y,MA X,WANG Z,et al.User-guided deep anime line art colorization with conditional adversarial networks[C]//26th ACM Multimedia Conference.ACM,2018:1536-1544.
[16] LU Q W,TAO Q C,ZHAO Y L,et al.Sketch SimplificationUsing Generative Adversarial Networks[J].Acat Automatica Sinica,2018.5(44):840-854.
[17] BAO R D,YU H,ZHU D F,et al.Automatic Makeup with Region Sensitive Generative Adversarial Networks[J].Journal of Software,2019,30(4):896-913.
[18] MAO X,LI Q,XIE H,et al.Least Squares Generative Adversarial Networks[C]//International Conference on Computer Vision(ICCV).IEEE,2017:2813-2821.
[19] ARJOVSKY M,BOTTOU L.Towards principled methods for training generative adversarial networks [C]//International Conference on Learning Representations.ICLR,2019.
[20] SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognitio [C]//International Conference on Learning Representations.ICLR,2015.
[21] HONG Y,HWANG U,YOO J,et al.How generative adversarialnetworks and their variants work:An overview[J].ACM Computing Surveys,2019,52(1):Article 10.
[22] KRIZHEVSKY A,SUTSKEVER I,HINTON G E.ImageNet classification with deep convolutional neural networks[C]//Conference on Neural Information Processing Systems.MIT Press,2012:1097-1105.
[23] IOFFE S,SZEGEDY C.Batch normalization:Accelerating deep network training by reducing internal covariate shift[C]//International Conference on Machine Learning.ACM,2015:448-456.
[24] HEUSEL M,RAMSAUER H,UNTERTHINER T,et al. GANstrained by a two time-scale update rule converge to a local Nash equilibrium[C]//Conference on Neural Information Processing Systems.MIT Press,2017:6627-6638.
[1] XU Guo-ning, CHEN Yi-peng, CHEN Yi-ming, CHEN Jin-yin, WEN Hao. Data Debiasing Method Based on Constrained Optimized Generative Adversarial Networks [J]. Computer Science, 2022, 49(6A): 184-190.
[2] XU Hui, KANG Jin-meng, ZHANG Jia-wan. Digital Mural Inpainting Method Based on Feature Perception [J]. Computer Science, 2022, 49(6): 217-223.
[3] DOU Zhi, WANG Ning, WANG Shi-jie, WANG Zhi-hui, LI Hao-jie. Sketch Colorization Method with Drawing Prior [J]. Computer Science, 2022, 49(4): 195-202.
[4] GAO Zhi-yu, WANG Tian-jing, WANG Yue, SHEN Hang, BAI Guang-wei. Traffic Prediction Method for 5G Network Based on Generative Adversarial Network [J]. Computer Science, 2022, 49(4): 321-328.
[5] LI Si-quan, WAN Yong-jing, JIANG Cui-ling. Multiple Fundamental Frequency Estimation Algorithm Based on Generative Adversarial Networks for Image Removal [J]. Computer Science, 2022, 49(3): 179-184.
[6] SHI Da, LU Tian-liang, DU Yan-hui, ZHANG Jian-ling, BAO Yu-xuan. Generation Model of Gender-forged Face Image Based on Improved CycleGAN [J]. Computer Science, 2022, 49(2): 31-39.
[7] TAN Xin-yue, HE Xiao-hai, WANG Zheng-yong, LUO Xiao-dong, QING Lin-bo. Text-to-Image Generation Technology Based on Transformer Cross Attention [J]. Computer Science, 2022, 49(2): 107-115.
[8] ZHANG Wei-qi, TANG Yi-feng, LI Lin-yan, HU Fu-yuan. Image Stream From Paragraph Method Based on Scene Graph [J]. Computer Science, 2022, 49(1): 233-240.
[9] LIN Zhen-xian, ZHANG Meng-kai, WU Cheng-mao, ZHENG Xing-ning. Face Image Inpainting with Generative Adversarial Network [J]. Computer Science, 2021, 48(9): 174-180.
[10] XU Tao, TIAN Chong-yang, LIU Cai-hua. Deep Learning for Abnormal Crowd Behavior Detection:A Review [J]. Computer Science, 2021, 48(9): 125-134.
[11] PAN Xiao-qin, LU Tian-liang, DU Yan-hui, TONG Xin. Overview of Speech Synthesis and Voice Conversion Technology Based on Deep Learning [J]. Computer Science, 2021, 48(8): 200-208.
[12] YE Hong-liang, ZHU Wan-ning, HONG Lei. Music Style Transfer Method with Human Voice Based on CQT and Mel-spectrum [J]. Computer Science, 2021, 48(6A): 326-330.
[13] WANG Jian-ming, LI Xiang-feng, YE Lei, ZUO Dun-wen, ZHANG Li-ping. Medical Image Deblur Using Generative Adversarial Networks with Channel Attention [J]. Computer Science, 2021, 48(6A): 101-106.
[14] HU Yu-jie, CHANG Jian-hui, ZHANG Jian. Image Synthesis with Semantic Region Style Constraint [J]. Computer Science, 2021, 48(2): 134-141.
[15] YU Xiao-ming, HUANG Hua. Research on Application of Improved GAN Network in Generating Short Video [J]. Computer Science, 2021, 48(11A): 625-629.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!