Computer Science ›› 2018, Vol. 45 ›› Issue (12): 229-234.doi: 10.11896/j.issn.1002-137X.2018.12.038

• Graphics, Image & Pattern Recognition • Previous Articles     Next Articles

Image Inpainting Based on Generative Adversarial Networks

SUN Quan, ZENG Xiao-qin   

  1. (Institute of Intelligence Science and Technology,College of Computer and Information,Hohai University,Nanjing 211100,China)
  • Received:2017-11-01 Online:2018-12-15 Published:2019-02-25

Abstract: Aiming at the problems of the restricted shape and size of the damaged area,the obvious inpainting tracks and the discontinuous inpainting edge in the existing image impainting algorithms,this paper proposed an image inpainting method based on generative adversarial networks.In this method,a new generative model named generative adversarial networks(GAN) is used as the basic framework with combining Wasserstein distance and the idea of conditional genera-tive adversarial networks(CGAN).The network receives the damaged image as additional conditional information and combines adversarial loss with content loss to train the network model for restoring pixels of missing areas.This method can be used to repair most of the damages in images.The experimental results on two datasets of CelebA and LFW suggest the capability of this method to obtain good performance.

Key words: Adversarial learning, Generative adversarial networks(GAN), Image inpainting, Wasserstein distance

CLC Number: 

  • TP391.41
[1]CHAN T F,SHEN J.Mathematical Models for Local Nontexture Inpaintings[J].Siam Journal on Applied Mathematics,2002,62(3):1019-1043.
[2]BARNES C,SHECHTMAN E,FINKELSTEIN A,et al.PatchMatch:a randomized correspondence algorithm for structural image editing[J].Acm Transactions on Graphics,2009,28(3):1-11.
[3]PATHAK D,KRAHENBUHL P,DONAHUE J,et al.Context Encoders:Feature Learning by Inpainting[C]∥Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE Computer Society,2016:2536-2544.
[4]GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al.Generative Adversarial Networks[C]∥Proceedings of the Conference on Advances in Neural Information Processing Systems.Montreal,Canada:Curran Associates,2014:2672-2680.
[5]DENTON E,GROSS S,FERGUS R.Semi-Supervised Learning with Context-Conditional Generative Adversarial Networks[J].arXiv preprint arXiv:1611.06430,2016.
[6]SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-Scale Image Recognition[J].arXiv preprint arXiv:1409.1556,2014.
[7]YEH R A,CHEN C,LIM T Y,et al.Semantic Image Inpainting with Deep Generative Models[C]∥Proceedings of IEEE Confe-rence on Computer Vision and Pattern Recognition.Honolulu,HI,USA:IEEE Computer Society,2017:6882-6890.
[8]RADFORD A,METZ L,CHINTALA S.Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks[J].arXiv preprint arXiv:1511.06434,2015.
[9]ARJOVSKY M,CHINTALA S,BOTTOU L.Wasserstein gan[J].arXiv preprint arXiv:1701.07875,2017.
[10]RONNEBERGER O,FISCHER P,BROX T.U-Net:Convolu-tional Networks for Biomedical Image Segmentation[C]∥Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention.Munich:Springer Cham,2015:234-241.
[11]WANG K F,GOU C,DUAN Y J,et al.Generative adversarial networks:the state of the art and beyond[J].Acta Automatica Sinica,2017,43(3):321-332.(in Chinese)
王坤峰,苟超,段艳杰,等.生成式对抗网络GAN 的研究进展与展望[J].自动化学报,2017,43(3):321-332.
[12]RATLIFF L J,BURDEN S A,SASTRY S S.Characterization and computation of local Nash equilibria in continuous games[C]∥Proceedings of the 2013 51st Annual Allerton Conference on Communication,Control,and Computing.Allerton:IEEE Computer Society,2013:917-924.
[13]MIRZA M,OSINDERO S.Conditional Generative AdversarialNets[J].arXiv preprint arXiv:1411.1784,2014.
[14]IOFFE S,SZEGEDY C.Batch Normalization:Accelerating Deep Network Training by Reducing Internal Covariate Shift[C]∥Proceedings of the 32nd International Conference on Machine Learning.Lille,France:International Machine Learning Society,2015:448-456.
[15]TREFNY J,MATAS J.Extended Set of Local Binary Patterns for Rapid Object Detection[C]∥Proceedings of the 15th Computer Vision Winter Workshop.Nove Hrady,Czech Republic:CVWW,2010:1-7.
[16]VIOLA P,JONES M.Rapid object detection using a boostedcascade of simple features[C]∥Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Kauai:IEEE Computer Society,2001:511-518.
[17]GULRAJANI I,AHMED F,ARJOVSKY M,et al.ImprovedTraining of Wasserstein GANs[C]∥Proceedings of Annual Conference on Neural Information Processing Systems.Long Beach,CA,USA,2017:5769-5779.
[18]KINGMA D P,BA J.Adam:A Method for Stochastic Optimization[J].arXiv preprint arXiv:1412.6980,2014.
[19]GANGNET M,BLAKE A.Poisson image editing[C]∥Procee-dings of the ACM SIGGRAPH.New York:Association for Computing Machinery,2003:313-318.
[1] CAO Xiao-wen, LIANG Mei-yu, LU Kang-kang. Fine-grained Semantic Reasoning Based Cross-media Dual-way Adversarial Hashing Learning Model [J]. Computer Science, 2022, 49(9): 123-131.
[2] DOU Zhi, WANG Ning, WANG Shi-jie, WANG Zhi-hui, LI Hao-jie. Sketch Colorization Method with Drawing Prior [J]. Computer Science, 2022, 49(4): 195-202.
[3] HOU Hong-xu, SUN Shuo, WU Nier. Survey of Mongolian-Chinese Neural Machine Translation [J]. Computer Science, 2022, 49(1): 31-40.
[4] LIN Zhen-xian, ZHANG Meng-kai, WU Cheng-mao, ZHENG Xing-ning. Face Image Inpainting with Generative Adversarial Network [J]. Computer Science, 2021, 48(9): 174-180.
[5] LIU Li-bo, GOU Ting-ting. Cross-modal Retrieval Combining Deep Canonical Correlation Analysis and Adversarial Learning [J]. Computer Science, 2021, 48(9): 200-207.
[6] WANG Sheng, ZHANG Yang-sen, CHEN Ruo-yu, XIANG Ga. Text Matching Method Based on Fine-grained Difference Features [J]. Computer Science, 2021, 48(8): 60-65.
[7] ZHAN Wan-jiang, HONG Zhi-lin, FANG Lu-ping, WU Zhe-fu, LYU Yue-hua. Collaborative Filtering Recommendation Algorithm Based on Adversarial Learning [J]. Computer Science, 2021, 48(7): 172-177.
[8] ZHAO Lu-lu, SHEN Ling, HONG Ri-chang. Survey on Image Inpainting Research Progress [J]. Computer Science, 2021, 48(3): 14-26.
[9] LIU Lang, LI Liang, DAN Yuan-hong. Coherent Semantic Spatial-Temporal Attention Network for Video Inpainting [J]. Computer Science, 2021, 48(10): 239-245.
[10] MENG Li-sha, REN Kun, FAN Chun-qi, HUANG Long. Dense Convolution Generative Adversarial Networks Based Image Inpainting [J]. Computer Science, 2020, 47(8): 202-207.
[11] ZHOU Xian-chun, XU Yan. Adaptive Image Inpainting Based on Structural Correlation [J]. Computer Science, 2020, 47(4): 131-135.
[12] TANG Hao-feng, DONG Yuan-fang, ZHANG Yi-tong, SUN Juan-juan. Survey of Image Inpainting Algorithms Based on Deep Learning [J]. Computer Science, 2020, 47(11A): 151-164.
[13] YAO Zhe-wei, YANG Feng, HUANG Jing, LIU Ya-qin. Improved CycleGANs for Intravascular Ultrasound Image Enhancement [J]. Computer Science, 2019, 46(5): 221-227.
[14] GAN Ling, ZHAO Fu-chao, YANG Meng. Self-adaptive Group Sparse Representation Method for Image Inpainting [J]. Computer Science, 2018, 45(8): 272-276.
[15] ZHANG Lei and KANG Bao-sheng. Image Inpainting for Object Removal Based on Structure Sparsity and Patch Difference [J]. Computer Science, 2018, 45(5): 255-259.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!