Computer Science ›› 2018, Vol. 45 ›› Issue (5): 255-259.doi: 10.11896/j.issn.1002-137X.2018.05.044

Previous Articles     Next Articles

Image Inpainting for Object Removal Based on Structure Sparsity and Patch Difference

ZHANG Lei and KANG Bao-sheng   

  • Online:2018-05-15 Published:2018-07-25

Abstract: Aiming at the problems of unreasonable filling order and the mismatch error in the image inpainting for object removal,an image inpainting method based on structure sparsity and patch difference was proposed.Firstly,the structure sparsity of the patch is added in the priority computation,because not only the priority depends on the geometric characteristics of target patch,but also its neighborhood characteristics are reflected,which can improve the identification of the regional characteristics of target patch,so that the filling order is more reasonable.Secondly,the difference between the target patch and the exemplar patch is defined,and the new matching rule is defined on the basis of this.In the new matching rule,it not only measures the similarity degree between existing pixels,but also measures the difference degree between existing pixels and filled pixels,thus effectively preventing mismatch error and error accumulation.Experimental results show that the proposed method can effectively improve the restoration effect,and make the restored images more consistent with the visual consistency requirements.

Key words: Structure sparsity,Patch difference,Object removal,Image inpainting

[1] GUILLEMOT C,LE M O.Image Inpainting:Overview and Recent Advances [J].IEEE Signal Processing Magazine,2014,31(1):127-144.
[2] NIE H Y,ZHAI D H,YU J,et al.Large Damaged Area Image Inpainting Algorithm Based on Matching Model for Broken Structure [J].Computer Science,2016,43(10):292-296.(in Chinese) 聂洪玉,翟东海,鱼江,等.基于断裂结构线匹配模型的大破损区域图像修复算法[J].计算机科学,2016,43(10):292-296.
[3] WANG W.Study on Exemplar-based Tangka Image Inpainting Technologies [D].Chengdu:Southwest Jiaotong University,2016.(in Chinese) 王威.基于样本块的唐卡图像修复技术研究[D].成都:西南交通大学,2016.
[4] BHARDWAJ N,AGARWAL S.Review of Image Defect Detection and Inpainting Techniques and Scope for Improvements [J].International Journal of Emerging Sciences,2015,5(1):49-67.
[5] ZHANG L,KANG B,LIU B,et al.A New Inpainting Method for Object Removal based on Patch Local Feature and Sparse Representation [J].International Journal of Innovative Computing,Information and Control,2016,12(1):113-124.
[6] M ULLER J S.A Coupled Variational Problem of LinearGrowth Related to the Denoising and Inpainting of Images[J].Journal of Mathematical Sciences,2017,224(2):1-26.
[7] BERTALMIO M,SAPIRO G,CASELLES V,et al.Image inpainting[C]∥27th Annual Conference on Computer Graphics and Interactive Techniques.ACM Press/Addison-Wesley Publishing Co.,2000:417-424.
[8] CHAN T F,SHEN J,ZHOU H M.Total Variation Wavelet Inpainting [J].Journal of Mathematical imaging and Vision,2006,25(1):107-125.
[9] CHAN T F,SHEN J.Nontexture Inpainting by Curvature-dri-ven Diffusions[J].Journal of Visual Communication and Image Representation,2001,12(4):436-449.
[10] ZOU W G,ZHOU Z H,WANG Y.Image Inpainting Techno-logy Based on Non-subsampled Contourlet Transform [J].Journal of Computer Applications,2017,37(2):553-558.(in Chinese) 邹玮刚,周志辉,王洋.基于非降采样轮廓波变换的图像修复算法[J].计算机应用,2017,37(2):553-558.
[11] HUANG Y,LI K,YANG M.An Improved Image Inpainting Algorithm Based on Image Segmentation [J].Procedia Compu-ter Science,2017,107:796-801.
[12] CRIMINISI A,PEREZ P,TOYAMA K.Region Filling and Object Removal by Exemplar-based Image Inpainting [J].IEEE Transactions on Image Processing,2004,13(9):1200-1212.
[13] WONG A,ORCHARD J.A Nonlocal-means Approach to Exemplar-based Inpainting[C]∥15th IEEE International Conference on Image Processing.IEEE,2008:2600-2603.
[14] EFROS A A,LEUNG T K.Texture Synthesis by Non-parametric Sampling [C]∥IEEE International Conference on Computer Vision.1999:1033-1038.
[15] DENG L J,HUANG T Z,ZHAO X L.Exemplar-Based Image Inpainting Using a Modified Priority Definition [J].PLOS one,2015,10(10):1-18.
[16] ALILOU V K,YAGHMAEE F.Exemplar-based Image Inpain-ting Using Svd-based Approximation Matrix and Multi-scale Analysis[J].Multimedia Tools and Applications,2017,76(5):7213-7234.
[17] ZHANG Z,XU Y,YANG J,et al.A Survey of Sparse Representation:Algorithms and Applications[J].IEEE Access,2017,3:490-530.
[18] ZHANG L,KANG B,LIU B,et al.Image Inpainting Based on Exemplar and Sparse Representation [J].International Journal of Signal Processing,Image Processingand Pattern Recognition,2016,9(9):177-188.
[19] AHARON M,ELAD M,BRUCKSTEIN A.K-SVD:An Algorithm for Designing Over-complete Dictionaries for Sparse Representation [J].IEEE Transactions on Signal Processing,2006,54(11):4311-4322.
[20] ELAD M,STARCK J L,QUERRE P,et al.Simultaneous Cartoon and Texture Image Inpainting Using Morphological Component Analysis(MCA)[J].Applied and Computational Harmonic Analysis,2005,19(3):340-358.
[21] DAI L,JIANG D H,DING B,et al.Improved Digital Image Restoration Algorithm Based on Criminisi[J].Journal of Digital Information Management,2016,14(5):302-310.
[22] ZHANG S H,WANG K G,ZHU X.Improved Criminisi Algorithm Constrained by Local Feature[J].Computer Engineering and Applications,2014,50(8):127-130.(in Chinese) 张申华,王克刚,祝轩.局部特征信息约束的改进Criminisi算法[J].计算机工程与应用,2014,50(8):127-130.
[23] LIU Y F,WANG F L,XI X Y.Enhanced Algorithm for Exemplar-based Image Inpainting [C]∥2013 9th International Conference on Computational Intelligence and Security.IEEE,2013:209-213.
[24] NAN A,XI X.An Improved CriminisiAlgorithm Based on ANew Priority Function and Updating Confidence[C]∥International Conference on Biomedical Engineering and Informatics.IEEE,2014:885-889.
[25] WANG J,LU K,PAN D,et al.Letters:Robust Object Removal with An Exemplar-based Image InpaintingApproach[J].Neurocomputing,2014,123:150-155.
[26] XI X,WANG F,LIU Y.Improved Criminisi Algorithm Based on a New Priority Function With the Gray Entropy [C]∥2013 9th International Conference on Computational Intelligence and Security.IEEE,2013:214-218.
[27] LIU Y F,WANG F L,XI X Y,et al.Improved Algorithm for Image Inpainting Based on Texture Synthesis [J].Journal of Chinese Computer Systems,2014,35(12):2754-2758.(in Chinese) 刘业妃,王福龙,奚祥艳,等.改进的Criminisi图像修复算法[J].小型微型计算机系统,2014,35(12):2754-2758.
[28] XU Z,SUN J.Image Inpainting by Patch Propagation Using Patch Sparsity [J].IEEE Transactions on Image Processing,2010,19(5):1153-1165.
[29] ARBELAEZ P,MAIRE M,FOWLKES C,et al.Contour Detection and Hierarchical Image Segmentation [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2011,33(5):898-916.

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!