Computer Science

Computer Science ›› 2020, Vol. 47 ›› Issue (6A): 230-236.doi: 10.11896/JsJkx.190400118

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Face Image Restoration Based on Residual Generative Adversarial Network

LI Ze-wen, LI Zi-ming, FEI Tian-lu, WANG Rui-lin and XIE Zai-peng   

  1. College of Computer and Information,Hohai University,NanJing 211100,China
  • Published:2020-07-07
  • About author:LI Ze-wen, born in 1998, senior undergraduate.His main research interests include computer vision and distributed machine learning.
    XIE Zai-peng, born in 1982, associate professor.His main research interests include distributed and embedded machine learning.
  • Supported by:
    This work was supported by the National College Students’ Innovation and Entrepreneurship Training Program (201810294106).

PDF (PC)

1014

Abstract: Benefiting from the rapid development of computer vision,face image restoration technology can only use the contour of the face to generate a complete face image.At present,many face restoration techniques based on convolutional neural networks and generative adversarial networks have been proposed.They can restore partial damaged face images or even directly generate face images using face contours.However,the results of qualitative and quantitative analysis of face images restored by these techniques are not ideal,and there are many limitations in the restoration process.Therefore,this paper proposes a face image restoration method based on residual generative adversarial network (FR-RGAN),which improves the performance of the model by means of deep convolution,residual network and smaller convolution kernels,and restores the local details of the face by using the contour of the face,making it more vivid.Experimental results show that,compared with pix2pix,FR-RGAN has an improvement of 8.7%,2.1% and 9.6% respectively in mean square error,peak signal to noise ratio and structural similarity index,and 53.4%,12.6% and 30.1% better than non-residual method.

Key words: Computer vision, Face image restoration, Generative adversarial networks, Residual neural networks

CLC Number: 

  • TP183
[1] SHEN J,CHAN T F.Mathematical models for local nontextureinpaintings.SIAM Journal on Applied Mathematics,2002,62(3):1019-1043.
[2] BARNES C,SHECHTMAN E,FINKELSTEIN A,et al.PatchMatch:A randomized correspondence algorithm for structural image editing//ACM Transactions on Graphics (ToG).ACM,2009,28(3):24.
[3] LI Y,LIU S,YANG J,et al.Generative face completion//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:3911-3919.
[4] YEH R A,CHEN C,YIAN LIM T,et al.Semantic image inpainting with deep generative models//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:5485-5493.
[5] GAUTHIER J.Conditional generative adversarial nets for convolutional face generation.Class ProJect for Stanford CS231N:Convolutional Neural Networks for Visual Recognition,Winter semester,2014,2014(5):2.
[6] ISOLA P,ZHU J Y,ZHOU T,et al.Image-to-image translation with conditional adversarial networks//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:1125-1134.
[7] HAYS J,EFROS A A.Scene completion using millions of photographs.ACM Transactions on Graphics (TOG),2007,26(3):4.
[8] PATHAK D,KRAHENBUHL P,DONAHUE J,et al.Context encoders:Feature learning by inpainting//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:2536-2544.
[9] KINGMA D P,WELLING M.Auto-encoding variational bayes .arXiv:1312.6114,2013.
[10] LARSEN A B L,SNDERBY S K,LAROCHELLE H,et al.Autoencoding beyond pixels using a learned similarity metric.arXiv:1512.09300,2015.
[11] GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Generative adversarial nets//Advances in Neural Information Processing Systems.2014:2672-2680.
[12] DENTON E L,CHINTALA S,FERGUS R.Deep generative image models using a laplacian pyramid of adversarial networks//Advances in Neural Information Processing Systems.2015:1486-1494.
[13] RADFORD A,METZ L,CHINTALA S.Unsupervised repre sentation learning with deep convolutional generative adversarial networks.arXiv:1511.06434,2015.
[14] SUN Q,ZENG X Q.Image Inpainting Based on Generative Adversarial Networks .Computer Science,2018,45(12):229-234,261.
[15] CHENG X Y,XIE L,ZHU J X,et al.Review of Generative Adversarial Network .Computer Science,2019,46(3):74-81.
[16] XU Q,ZHONG S P,CHEN K Z,et al.Optimized Selection Method of Cycle-consistent Loss Coefficient of CycleGAN in Image Generation with Different Texture Complexity.Computer Science,2019,46(1):100-106.
[17] LIU F,LI Z W,et al.A Text-Based CAPTCHA Cracking System with Generative Adversarial Networks//2018 IEEE International Symposium on Multimedia (ISM).IEEE.2018.
[18] REED S,AKATA Z,YAN X,et al.Generative adversarial text to image synthesis.arXiv:1605.05396.
[19] ZHU J Y,PARK T,ISOLA P,et al.Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks.International Conference on Computer Vision,2017:2242-2251.
[20] BERTHELOT D,SCHUMM T,Metz L,et al.BEGAN:Boundary Equilibrium Generative Adversarial Networks.arXiv:Learning,2017.
[21] KING D E.Dlib-ml:A machine learning toolkit .Journal of Machine Learning Research,2009:1755-1758.
[22] KRIZHEVSKY A,ILYA S,GEOFFREY E H.Imagenet classification with deep convolutional neural networks//Advances in neural information processing systems.2012.
[23] HE K M,et al.Deep residual learning for image recognition //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016.
[24] IOFFE S,CHRISTIAN S.Batch normalization:Accelerating deep network training by reducing internal covariate shift .arXiv:1502.03167.
[25] XU B,WANG N,CHEN T,et al.Empirical evaluation of rectified activations in convolutional network .arXiv:1505.00853,2015.
[26] NAIR V,GEOFFREY E H.Rectified linear units improve restricted boltzmann machines//Proceedings of the 27th International Conference on Machine Learning.2010. man-talking .https://www.pexels.com/video/man-talking-1769632/.
[27] Putin delivers annual address to Russia’s Federal Assembly .https://www.youtube.com/watch?v=P6HM9pKrxqE.
[28] CHANNEL 90 seconds TV.Official Channel,President Trump speech to the 72nd Session of the UN General Assembly ..https://www.youtube.com/watch?v=pyZ965-3qP4&t=3957s.
[29] HUYNH-THU Q,MOHAMMED G.Scope of validity of PSNR in image/video quality assessment .Electronics Letters,2008:800-801.
[30] WANG Z.The SSIM index for image quality assessment.https://ece.uwaterloo.ca/~ z70wang/research/ssim.
[31] HORE A,DIEMEL Z.Image quality metrics:PSNR vs. SSIM//2010 20th International Conference on Pattern Recognition.IEEE,2010.
[1] XU Guo-ning, CHEN Yi-peng, CHEN Yi-ming, CHEN Jin-yin, WEN Hao. Data Debiasing Method Based on Constrained Optimized Generative Adversarial Networks [J]. Computer Science, 2022, 49(6A): 184-190.
[2] XU Hui, KANG Jin-meng, ZHANG Jia-wan. Digital Mural Inpainting Method Based on Feature Perception [J]. Computer Science, 2022, 49(6): 217-223.
[3] ZHANG Ji-kai, LI Qi, WANG Yue-ming, LYU Xiao-qi. Survey of 3D Gesture Tracking Algorithms Based on Monocular RGB Images [J]. Computer Science, 2022, 49(4): 174-187.
[4] DOU Zhi, WANG Ning, WANG Shi-jie, WANG Zhi-hui, LI Hao-jie. Sketch Colorization Method with Drawing Prior [J]. Computer Science, 2022, 49(4): 195-202.
[5] GAO Zhi-yu, WANG Tian-jing, WANG Yue, SHEN Hang, BAI Guang-wei. Traffic Prediction Method for 5G Network Based on Generative Adversarial Network [J]. Computer Science, 2022, 49(4): 321-328.
[6] LI Si-quan, WAN Yong-jing, JIANG Cui-ling. Multiple Fundamental Frequency Estimation Algorithm Based on Generative Adversarial Networks for Image Removal [J]. Computer Science, 2022, 49(3): 179-184.
[7] TAN Xin-yue, HE Xiao-hai, WANG Zheng-yong, LUO Xiao-dong, QING Lin-bo. Text-to-Image Generation Technology Based on Transformer Cross Attention [J]. Computer Science, 2022, 49(2): 107-115.
[8] GAN Chuang, WU Gui-xing, ZHAN Qing-yuan, WANG Peng-kun, PENG Zhi-lei. Multi-scale Gated Graph Convolutional Network for Skeleton-based Action Recognition [J]. Computer Science, 2022, 49(1): 181-186.
[9] ZHANG Wei-qi, TANG Yi-feng, LI Lin-yan, HU Fu-yuan. Image Stream From Paragraph Method Based on Scene Graph [J]. Computer Science, 2022, 49(1): 233-240.
[10] XU Tao, TIAN Chong-yang, LIU Cai-hua. Deep Learning for Abnormal Crowd Behavior Detection:A Review [J]. Computer Science, 2021, 48(9): 125-134.
[11] LIN Zhen-xian, ZHANG Meng-kai, WU Cheng-mao, ZHENG Xing-ning. Face Image Inpainting with Generative Adversarial Network [J]. Computer Science, 2021, 48(9): 174-180.
[12] PAN Xiao-qin, LU Tian-liang, DU Yan-hui, TONG Xin. Overview of Speech Synthesis and Voice Conversion Technology Based on Deep Learning [J]. Computer Science, 2021, 48(8): 200-208.
[13] YE Hong-liang, ZHU Wan-ning, HONG Lei. Music Style Transfer Method with Human Voice Based on CQT and Mel-spectrum [J]. Computer Science, 2021, 48(6A): 326-330.
[14] FENG Fu-rong, ZHANG Zhao-gong. Recent Advances for Object Contour Detection Technology [J]. Computer Science, 2021, 48(6A): 1-9.
[15] WANG Jian-ming, LI Xiang-feng, YE Lei, ZUO Dun-wen, ZHANG Li-ping. Medical Image Deblur Using Generative Adversarial Networks with Channel Attention [J]. Computer Science, 2021, 48(6A): 101-106.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.