计算机科学 ›› 2021, Vol. 48 ›› Issue (9): 174-180.doi: 10.11896/jsjkx.200800014
林椹尠1, 张梦凯2, 吴成茂3, 郑兴宁2
LIN Zhen-xian1, ZHANG Meng-kai2, WU Cheng-mao3, ZHENG Xing-ning2
摘要: 人脸图像修复技术是近年来图像处理领域的研究热点,而人脸图像大面积缺失导致损失语义信息过多,一直是该领域的重点难点问题。针对这一问题,文中提出了一种基于生成对抗网络的图像分步补全算法。将人脸图像修复问题分为两步,设计两个串联的生成对抗网络,首先残缺图像通过预补全网络进行图像的预补全,预补全图像进入增强网络进行特征增强;判别器分别判断预补全图像和增强图像与理想图像的差异性;采用长短时记忆单元连接两部分的信息流,增强信息的传递。然后使用内容损失、对抗损失和全变分损失相结合的损失函数,提高网络的修复效果。最后在CelebA数据集上进行实验,结果显示,所提算法相较于对比算法在峰值信噪比指标上提高了16.84%~22.85%,在结构相似性指标上提高了10%~12.82%。
中图分类号:
[1]HE Y T,TANG X H,ZHANG Y,et al.Improved Criminisi algorithm based on structure tensor[J].Journal of Image and Graphics,2018,23(10):1492-1507. [2]LAN X L,LIU H X,YAO H B.Improved image inpainting algorithm based on texture blocks and gradient feature[J].Computer Engineering and Applications,2018,54(20):172-177. [3]KRHENBÜHL P,KOLTUN V.Efficient Inference in FullyConnected CRFs with Gaussian Edge Potentials[J].Advances in Neural Information Processing Systems,2012,24(2011):109-117. [4]XU L M,WU Y J,ZHANG B.Image Inpainting AlgorithmBased on Adaptive High Order Variation in Eight Neighbors[J].Journal of Graphics,2017,38(4):558-565. [5]BERTALMIO M,SAPIRO G,CASELLES V,et al.Image in-painting[C]//Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques.ACM Press/Addison-Wesley Publishing Co.,2000:417-424. [6]LEVIN A,ZOMET A,WEISS Y.Learning how to inpaint from global mage statistics[C]//IEEE In International Conference on Computer Vision.2003:305-312. [7]SHEN J,CHAN T F.Mathematical Models for Local Nontexture Inpaintings[J].SIAM Journal on Applied Mathematics,2001,62:1019-1043. [8]SHEN J,KANG S H,CHAN T F.Euler's Elastica and Curvature-Based Inpainting[J].SIAM Journal on AppliedMathema-tics,2002,63(2):564-592. [9]BOUREAU Y L,BACH F,LECUN Y,et al.Learning mid-level features for recognition[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.IEEE,2010,2559-2566. [10]LI S,ZHAO M.Image inpainting with salient structure completion and texture propagation[J].Pattern Recognition Letters,2011,32(9):1256-1266. [11]EFROS A A,LEUNG T K.Texture synthesis by non-parame-tric sampling[C]//Proceedings of the Seventh IEEE Internatio-nal Conference on Computer Vision.1999,2:1033-1038. [12]CRIMINISI A,PÉREZ P,TOYAMA K.Region filling and object removal by exemplar-based image inpainting[J].IEEE Transactions on Image Processing,2004,13(9):1200-1212. [13]BARNES C,SHECHTMAN E,FINKELSTEIN A,et al.PatchMatch:A randomized correspondence algorithm for structural image editing[C]//ACM Transactions on Graphics (ToG).ACM,2009:1-11. [14]WEI Y,LIU S.Domain-based structure-aware image inpainting[J].Signal Image & Video Processing,2016,10(5):911-919. [15]GIRSHICK R,DONAHUE J,DARRELL T,et al.Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2014:580-587. [16]LONG J,SHELHAMER E,DARRELL T.Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:3431-3440. [17]XIE J,XU L,CHEN E.Image denoising and inpainting withdeep neural networks[C]//Advances in Neural Information Processing Systems.2012:341-349. [18]KÖHLER R,SCHULER C,SCHÖLKOPF B,et al.Mask.specific inpainting with deep neural networks[C]//German Confe-rence on Pattern Recognition.Cham:Springer,2014:523-534. [19]GOODFELLOW I,POUGET-ABADIE J,MIRZAM,et al.Gene-rative adversarial nets[C]//Advances in Neural Information Processing Systems.2014:2672-2680. [20]PATHAK D,KRAHENBUHL P,DONAHUE J,et al.Context encoders:Feature learning by inpainting[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:2536-2544. [21]YANG C,LU X,LIN Z,et al.High-resolution image inpainting using multi-scale neural patch synthesis[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:6721-6729. [22]IIZUKA S S,SERRA E,SHIKAWA H.Globally and locally consistent image completion[J].ACM Transactions on Gra-phics (TOG),2017,36(4):107:1-2. [23]YU J H,ZHE L,YANG J M,et al.Generative Image Inpainting with Contextual Attention [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.2018:5505-5514. [24]LIU G,REDA F A,SHIH K J,et al.Image inpainting for irre-gular holes using partial convolutions [C]//Proceedings of the European Conference on Computer Vision (ECCV).2018:85-100. [25]HOCHREITER S,SCHMIDHUBER J.Long Short-Term Me-mory[J].Neural Computation,1997,9(8):1735-1780. [26]YEH R A,CHEN C,LIM T Y,et al.Semantic Image Inpainting with Deep Generative Models [C]//CVPR 2016.2016:5485-5493. [27]ISOLA P,ZHU J Y,ZHOU T H,et al.Image-to-image translation with conditional adversarial networks[C]//IEEEConfe-rence on Computer Vision and Pattern Recognition (CVPR).2017. [28]RONNEBERGER O,FISCHER P,BROX T.U-Net:Convolu-tional Networks for Biomedical Image Segmentation[C]//International Conference on Medical Image Computing and Computer-assisted Intervention.Cham:Springer,2015:234-241. [29]RADFORD A,METZ L,CHINTALA S.Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks [C]//Proceedings of the International Confe-rence on Learning Representations (ICLR).2016. [30]ZHAO H,GALLO O,FROSIO I,et al.Loss functions for image restoration with neural networks[J].IEEE Transactions on Computational Imaging,2016,3(1):47-57. [31]WHYTE O,SIVIC J,ZISSERMAN A,et al.Non-uniform deblurring for shaken images[J].International Journal of Compu-ter Vision,2012,98(2):168-186. [32]RUDIN L I,OSHER S,FATEMI E.Nonlinear total variation based noise removal algorithms[J].Physica D Nonlinear Phenomena,1992,60(1/2/3/4):259-268. [33]NAH S,KIM T H,LEE K M.Deep multi-scale convolutional neural network for dynamic scene deblurring[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:3883-3891. [34]TONG Y B,ZHANG Q S,QI Y P.Image Quality Assessing by Combining PSNR with SSIM[J].Journal of Image and Gra-phics,2006,11(12),1758-1763. [35]WANG Z,BOVIK A C,SHEIKH H R,et al.Image quality as-sessment:from error visibility to structural similarity[J].IEEE Transactions on Image Processing,2004,13(4):600-612. [36]ZHENG C X,CHAM T J,CAI J.Pluralistic Image Completion [C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).2019:1438-1447. [37]CIMPOI M,MAJI S,KOKKINOS I,et al.Describing Textures in the Wild[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).2014:3606-3613. |
[1] | 饶志双, 贾真, 张凡, 李天瑞. 基于Key-Value关联记忆网络的知识图谱问答方法 Key-Value Relational Memory Networks for Question Answering over Knowledge Graph 计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277 |
[2] | 汤凌韬, 王迪, 张鲁飞, 刘盛云. 基于安全多方计算和差分隐私的联邦学习方案 Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy 计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108 |
[3] | 张佳, 董守斌. 基于评论方面级用户偏好迁移的跨领域推荐算法 Cross-domain Recommendation Based on Review Aspect-level User Preference Transfer 计算机科学, 2022, 49(9): 41-47. https://doi.org/10.11896/jsjkx.220200131 |
[4] | 徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺. 时序知识图谱表示学习 Temporal Knowledge Graph Representation Learning 计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204 |
[5] | 王剑, 彭雨琦, 赵宇斐, 杨健. 基于深度学习的社交网络舆情信息抽取方法综述 Survey of Social Network Public Opinion Information Extraction Based on Deep Learning 计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099 |
[6] | 郝志荣, 陈龙, 黄嘉成. 面向文本分类的类别区分式通用对抗攻击方法 Class Discriminative Universal Adversarial Attack for Text Classification 计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077 |
[7] | 姜梦函, 李邵梅, 郑洪浩, 张建朋. 基于改进位置编码的谣言检测模型 Rumor Detection Model Based on Improved Position Embedding 计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046 |
[8] | 孙奇, 吉根林, 张杰. 基于非局部注意力生成对抗网络的视频异常事件检测方法 Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection 计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061 |
[9] | 胡艳羽, 赵龙, 董祥军. 一种用于癌症分类的两阶段深度特征选择提取算法 Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification 计算机科学, 2022, 49(7): 73-78. https://doi.org/10.11896/jsjkx.210500092 |
[10] | 戴朝霞, 李锦欣, 张向东, 徐旭, 梅林, 张亮. 基于DNGAN的磁共振图像超分辨率重建算法 Super-resolution Reconstruction of MRI Based on DNGAN 计算机科学, 2022, 49(7): 113-119. https://doi.org/10.11896/jsjkx.210600105 |
[11] | 程成, 降爱莲. 基于多路径特征提取的实时语义分割方法 Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction 计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157 |
[12] | 侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木. 中文预训练模型研究进展 Advances in Chinese Pre-training Models 计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018 |
[13] | 周慧, 施皓晨, 屠要峰, 黄圣君. 基于主动采样的深度鲁棒神经网络学习 Robust Deep Neural Network Learning Based on Active Sampling 计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044 |
[14] | 金方焱, 王秀利. 融合RACNN和BiLSTM的金融领域事件隐式因果关系抽取 Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM 计算机科学, 2022, 49(7): 179-186. https://doi.org/10.11896/jsjkx.210500190 |
[15] | 苏丹宁, 曹桂涛, 王燕楠, 王宏, 任赫. 小样本雷达辐射源识别的深度学习方法综述 Survey of Deep Learning for Radar Emitter Identification Based on Small Sample 计算机科学, 2022, 49(7): 226-235. https://doi.org/10.11896/jsjkx.210600138 |
|