基于残差生成对抗网络的人脸图像复原

doi:10.11896/JsJkx.190400118

摘要/Abstract

摘要： 得益于计算机视觉的快速发展,人脸图像复原技术可以仅利用人脸的轮廓来生成完整的人脸图像。目前已有许多基于卷积神经网络和生成对抗网络等方法的人脸复原技术被提出,它们可以利用部分破损的人脸图像进行复原或者使用人脸轮廓直接生成人脸图像。然而,使用这些技术复原后的人脸图像在定性和定量分析时效果不够理想,并且复原时存在诸多的条件限制。因此,文中提出了一种基于残差生成对抗网络的人脸图像复原(FR-RGAN)方法,该方法借助深度卷积、残差网络和更小的卷积核,提升了模型性能,利用人脸的轮廓复原面部局部细节,使其更加生动地呈现出来。实验结果表明,FR-RGAN在均方误差、峰值信噪比和结构相似度指标上比pix2pix分别提高了8.7%,2.1%和9.6%,比无残差方法分别提高了53.4%,12.6%和30.1%。

关键词: 残差网络, 计算机视觉, 人脸图像复原, 生成对抗网络

Abstract: Benefiting from the rapid development of computer vision,face image restoration technology can only use the contour of the face to generate a complete face image.At present,many face restoration techniques based on convolutional neural networks and generative adversarial networks have been proposed.They can restore partial damaged face images or even directly generate face images using face contours.However,the results of qualitative and quantitative analysis of face images restored by these techniques are not ideal,and there are many limitations in the restoration process.Therefore,this paper proposes a face image restoration method based on residual generative adversarial network (FR-RGAN),which improves the performance of the model by means of deep convolution,residual network and smaller convolution kernels,and restores the local details of the face by using the contour of the face,making it more vivid.Experimental results show that,compared with pix2pix,FR-RGAN has an improvement of 8.7%,2.1% and 9.6% respectively in mean square error,peak signal to noise ratio and structural similarity index,and 53.4%,12.6% and 30.1% better than non-residual method.

Key words: Computer vision, Face image restoration, Generative adversarial networks, Residual neural networks

中图分类号:

TP183

李泽文, 李子铭, 费天禄, 王瑞琳, 谢在鹏. 基于残差生成对抗网络的人脸图像复原[J]. 计算机科学, 2020, 47(6A): 230-236. https://doi.org/10.11896/JsJkx.190400118

LI Ze-wen, LI Zi-ming, FEI Tian-lu, WANG Rui-lin and XIE Zai-peng. Face Image Restoration Based on Residual Generative Adversarial Network[J]. Computer Science, 2020, 47(6A): 230-236. https://doi.org/10.11896/JsJkx.190400118

参考文献

[1] SHEN J,CHAN T F.Mathematical models for local nontextureinpaintings.SIAM Journal on Applied Mathematics,2002,62(3):1019-1043.
[2] BARNES C,SHECHTMAN E,FINKELSTEIN A,et al.PatchMatch:A randomized correspondence algorithm for structural image editing//ACM Transactions on Graphics (ToG).ACM,2009,28(3):24.
[3] LI Y,LIU S,YANG J,et al.Generative face completion//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:3911-3919.
[4] YEH R A,CHEN C,YIAN LIM T,et al.Semantic image inpainting with deep generative models//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:5485-5493.
[5] GAUTHIER J.Conditional generative adversarial nets for convolutional face generation.Class ProJect for Stanford CS231N:Convolutional Neural Networks for Visual Recognition,Winter semester,2014,2014(5):2.
[6] ISOLA P,ZHU J Y,ZHOU T,et al.Image-to-image translation with conditional adversarial networks//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:1125-1134.
[7] HAYS J,EFROS A A.Scene completion using millions of photographs.ACM Transactions on Graphics (TOG),2007,26(3):4.
[8] PATHAK D,KRAHENBUHL P,DONAHUE J,et al.Context encoders:Feature learning by inpainting//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:2536-2544.
[9] KINGMA D P,WELLING M.Auto-encoding variational bayes .arXiv:1312.6114,2013.
[10] LARSEN A B L,SNDERBY S K,LAROCHELLE H,et al.Autoencoding beyond pixels using a learned similarity metric.arXiv:1512.09300,2015.
[11] GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Generative adversarial nets//Advances in Neural Information Processing Systems.2014:2672-2680.
[12] DENTON E L,CHINTALA S,FERGUS R.Deep generative image models using a laplacian pyramid of adversarial networks//Advances in Neural Information Processing Systems.2015:1486-1494.
[13] RADFORD A,METZ L,CHINTALA S.Unsupervised repre sentation learning with deep convolutional generative adversarial networks.arXiv:1511.06434,2015.
[14] SUN Q,ZENG X Q.Image Inpainting Based on Generative Adversarial Networks .Computer Science,2018,45(12):229-234,261.
[15] CHENG X Y,XIE L,ZHU J X,et al.Review of Generative Adversarial Network .Computer Science,2019,46(3):74-81.
[16] XU Q,ZHONG S P,CHEN K Z,et al.Optimized Selection Method of Cycle-consistent Loss Coefficient of CycleGAN in Image Generation with Different Texture Complexity.Computer Science,2019,46(1):100-106.
[17] LIU F,LI Z W,et al.A Text-Based CAPTCHA Cracking System with Generative Adversarial Networks//2018 IEEE International Symposium on Multimedia (ISM).IEEE.2018.
[18] REED S,AKATA Z,YAN X,et al.Generative adversarial text to image synthesis.arXiv:1605.05396.
[19] ZHU J Y,PARK T,ISOLA P,et al.Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks.International Conference on Computer Vision,2017:2242-2251.
[20] BERTHELOT D,SCHUMM T,Metz L,et al.BEGAN:Boundary Equilibrium Generative Adversarial Networks.arXiv:Learning,2017.
[21] KING D E.Dlib-ml:A machine learning toolkit .Journal of Machine Learning Research,2009:1755-1758.
[22] KRIZHEVSKY A,ILYA S,GEOFFREY E H.Imagenet classification with deep convolutional neural networks//Advances in neural information processing systems.2012.
[23] HE K M,et al.Deep residual learning for image recognition //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016.
[24] IOFFE S,CHRISTIAN S.Batch normalization:Accelerating deep network training by reducing internal covariate shift .arXiv:1502.03167.
[25] XU B,WANG N,CHEN T,et al.Empirical evaluation of rectified activations in convolutional network .arXiv:1505.00853,2015.
[26] NAIR V,GEOFFREY E H.Rectified linear units improve restricted boltzmann machines//Proceedings of the 27th International Conference on Machine Learning.2010. man-talking .https://www.pexels.com/video/man-talking-1769632/.
[27] Putin delivers annual address to Russia’s Federal Assembly .https://www.youtube.com/watch?v=P6HM9pKrxqE.
[28] CHANNEL 90 seconds TV.Official Channel,President Trump speech to the 72nd Session of the UN General Assembly ..https://www.youtube.com/watch?v=pyZ965-3qP4&t=3957s.
[29] HUYNH-THU Q,MOHAMMED G.Scope of validity of PSNR in image/video quality assessment .Electronics Letters,2008:800-801.
[30] WANG Z.The SSIM index for image quality assessment.https://ece.uwaterloo.ca/~ z70wang/research/ssim.
[31] HORE A,DIEMEL Z.Image quality metrics:PSNR vs. SSIM//2010 20th International Conference on Pattern Recognition.IEEE,2010.

相关文章 15

[1]	张佳, 董守斌. 基于评论方面级用户偏好迁移的跨领域推荐算法 Cross-domain Recommendation Based on Review Aspect-level User Preference Transfer 计算机科学, 2022, 49(9): 41-47. https://doi.org/10.11896/jsjkx.220200131
[2]	王馨彤, 王璇, 孙知信. 基于多尺度记忆残差网络的网络流量异常检测模型 Network Traffic Anomaly Detection Method Based on Multi-scale Memory Residual Network 计算机科学, 2022, 49(8): 314-322. https://doi.org/10.11896/jsjkx.220200011
[3]	孙奇, 吉根林, 张杰. 基于非局部注意力生成对抗网络的视频异常事件检测方法 Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection 计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[4]	戴朝霞, 李锦欣, 张向东, 徐旭, 梅林, 张亮. 基于DNGAN的磁共振图像超分辨率重建算法 Super-resolution Reconstruction of MRI Based on DNGAN 计算机科学, 2022, 49(7): 113-119. https://doi.org/10.11896/jsjkx.210600105
[5]	高荣华, 白强, 王荣, 吴华瑞, 孙想. 改进注意力机制的多叉树网络多作物早期病害识别方法 Multi-tree Network Multi-crop Early Disease Recognition Method Based on Improved Attention Mechanism 计算机科学, 2022, 49(6A): 363-369. https://doi.org/10.11896/jsjkx.210500044
[6]	尹文兵, 高戈, 曾邦, 王霄, 陈怡. 基于时频域生成对抗网络的语音增强算法 Speech Enhancement Based on Time-Frequency Domain GAN 计算机科学, 2022, 49(6): 187-192. https://doi.org/10.11896/jsjkx.210500114
[7]	徐辉, 康金梦, 张加万. 基于特征感知的数字壁画复原方法 Digital Mural Inpainting Method Based on Feature Perception 计算机科学, 2022, 49(6): 217-223. https://doi.org/10.11896/jsjkx.210500105
[8]	韩红旗, 冉亚鑫, 张运良, 桂婕, 高雄, 易梦琳. 基于共同子空间分类学习的跨媒体检索研究 Study on Cross-media Information Retrieval Based on Common Subspace Classification Learning 计算机科学, 2022, 49(5): 33-42. https://doi.org/10.11896/jsjkx.210200157
[9]	赵人行, 徐频捷, 刘瑶. 基于深度卷积残差网络的心电单导联房颤检测方法 ECG-based Atrial Fibrillation Detection Based on Deep Convolutional Residual Neural Network 计算机科学, 2022, 49(5): 186-193. https://doi.org/10.11896/jsjkx.220200002
[10]	高心悦, 田汉民. 基于改进U-Net网络的液滴分割方法 Droplet Segmentation Method Based on Improved U-Net Network 计算机科学, 2022, 49(4): 227-232. https://doi.org/10.11896/jsjkx.210300193
[11]	张红民, 李萍萍, 房晓冰, 刘宏. 改进YOLOv3网络模型的人体异常行为检测方法 Human Abnormal Behavior Detection Method Based on Improved YOLOv3 Network Model 计算机科学, 2022, 49(4): 233-238. https://doi.org/10.11896/jsjkx.210300251
[12]	高志宇, 王天荆, 汪悦, 沈航, 白光伟. 基于生成对抗网络的5G网络流量预测方法 Traffic Prediction Method for 5G Network Based on Generative Adversarial Network 计算机科学, 2022, 49(4): 321-328. https://doi.org/10.11896/jsjkx.210300240
[13]	张继凯, 李琦, 王月明, 吕晓琪. 基于单目RGB图像的三维手势跟踪算法综述 Survey of 3D Gesture Tracking Algorithms Based on Monocular RGB Images 计算机科学, 2022, 49(4): 174-187. https://doi.org/10.11896/jsjkx.210700084
[14]	黎思泉, 万永菁, 蒋翠玲. 基于生成对抗网络去影像的多基频估计算法 Multiple Fundamental Frequency Estimation Algorithm Based on Generative Adversarial Networks for Image Removal 计算机科学, 2022, 49(3): 179-184. https://doi.org/10.11896/jsjkx.201200081
[15]	瞿中, 陈雯. 基于空洞卷积和多特征融合的混凝土路面裂缝检测 Concrete Pavement Crack Detection Based on Dilated Convolution and Multi-features Fusion 计算机科学, 2022, 49(3): 192-196. https://doi.org/10.11896/jsjkx.210100164

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed