基于结构稀疏度和块差异度的目标移除图像修复

doi:10.11896/j.issn.1002-137X.2018.05.044

摘要/Abstract

摘要： 针对目标移除修复方法中存在的修复顺序不合理以及错误匹配问题,提出一种基于结构稀疏度和块差异度的图像修复方法。首先,在优先权中增加块的结构稀疏度计算,使优先权不仅依赖于目标块的几何特征,而且可以反映其邻域特征,提高了对目标块所处区域特征的辨识度,从而使修复顺序更加合理。其次,定义目标块与样本块之间的差异度,并在此基础上定义新的匹配规则,不仅对已存在像素之间的相似程度进行衡量,而且对已存在像素与填充像素之间的差异程度进行衡量,从而有效防止错误匹配以及错误累积。实验结果表明,该方法可以有效提高图像的修复效果,使修复图像更加符合视觉一致性要求。

关键词: 结构稀疏度,块差异度,目标移除,图像修复

Abstract: Aiming at the problems of unreasonable filling order and the mismatch error in the image inpainting for object removal,an image inpainting method based on structure sparsity and patch difference was proposed.Firstly,the structure sparsity of the patch is added in the priority computation,because not only the priority depends on the geometric characteristics of target patch,but also its neighborhood characteristics are reflected,which can improve the identification of the regional characteristics of target patch,so that the filling order is more reasonable.Secondly,the difference between the target patch and the exemplar patch is defined,and the new matching rule is defined on the basis of this.In the new matching rule,it not only measures the similarity degree between existing pixels,but also measures the difference degree between existing pixels and filled pixels,thus effectively preventing mismatch error and error accumulation.Experimental results show that the proposed method can effectively improve the restoration effect,and make the restored images more consistent with the visual consistency requirements.

Key words: Structure sparsity,Patch difference,Object removal,Image inpainting

张雷,康宝生. 基于结构稀疏度和块差异度的目标移除图像修复[J]. 计算机科学, 2018, 45(5): 255-259. https://doi.org/10.11896/j.issn.1002-137X.2018.05.044

ZHANG Lei and KANG Bao-sheng. Image Inpainting for Object Removal Based on Structure Sparsity and Patch Difference[J]. Computer Science, 2018, 45(5): 255-259. https://doi.org/10.11896/j.issn.1002-137X.2018.05.044

参考文献

[1] GUILLEMOT C,LE M O.Image Inpainting:Overview and Recent Advances [J].IEEE Signal Processing Magazine,2014,31(1):127-144.
[2] NIE H Y,ZHAI D H,YU J,et al.Large Damaged Area Image Inpainting Algorithm Based on Matching Model for Broken Structure [J].Computer Science,2016,43(10):292-296.(in Chinese) 聂洪玉,翟东海,鱼江,等.基于断裂结构线匹配模型的大破损区域图像修复算法[J].计算机科学,2016,43(10):292-296.
[3] WANG W.Study on Exemplar-based Tangka Image Inpainting Technologies [D].Chengdu:Southwest Jiaotong University,2016.(in Chinese) 王威.基于样本块的唐卡图像修复技术研究[D].成都:西南交通大学,2016.
[4] BHARDWAJ N,AGARWAL S.Review of Image Defect Detection and Inpainting Techniques and Scope for Improvements [J].International Journal of Emerging Sciences,2015,5(1):49-67.
[5] ZHANG L,KANG B,LIU B,et al.A New Inpainting Method for Object Removal based on Patch Local Feature and Sparse Representation [J].International Journal of Innovative Computing,Information and Control,2016,12(1):113-124.
[6] M ULLER J S.A Coupled Variational Problem of LinearGrowth Related to the Denoising and Inpainting of Images[J].Journal of Mathematical Sciences,2017,224(2):1-26.
[7] BERTALMIO M,SAPIRO G,CASELLES V,et al.Image inpainting[C]∥27th Annual Conference on Computer Graphics and Interactive Techniques.ACM Press/Addison-Wesley Publishing Co.,2000:417-424.
[8] CHAN T F,SHEN J,ZHOU H M.Total Variation Wavelet Inpainting [J].Journal of Mathematical imaging and Vision,2006,25(1):107-125.
[9] CHAN T F,SHEN J.Nontexture Inpainting by Curvature-dri-ven Diffusions[J].Journal of Visual Communication and Image Representation,2001,12(4):436-449.
[10] ZOU W G,ZHOU Z H,WANG Y.Image Inpainting Techno-logy Based on Non-subsampled Contourlet Transform [J].Journal of Computer Applications,2017,37(2):553-558.(in Chinese) 邹玮刚,周志辉,王洋.基于非降采样轮廓波变换的图像修复算法[J].计算机应用,2017,37(2):553-558.
[11] HUANG Y,LI K,YANG M.An Improved Image Inpainting Algorithm Based on Image Segmentation [J].Procedia Compu-ter Science,2017,107:796-801.
[12] CRIMINISI A,PEREZ P,TOYAMA K.Region Filling and Object Removal by Exemplar-based Image Inpainting [J].IEEE Transactions on Image Processing,2004,13(9):1200-1212.
[13] WONG A,ORCHARD J.A Nonlocal-means Approach to Exemplar-based Inpainting[C]∥15th IEEE International Conference on Image Processing.IEEE,2008:2600-2603.
[14] EFROS A A,LEUNG T K.Texture Synthesis by Non-parametric Sampling [C]∥IEEE International Conference on Computer Vision.1999:1033-1038.
[15] DENG L J,HUANG T Z,ZHAO X L.Exemplar-Based Image Inpainting Using a Modified Priority Definition [J].PLOS one,2015,10(10):1-18.
[16] ALILOU V K,YAGHMAEE F.Exemplar-based Image Inpain-ting Using Svd-based Approximation Matrix and Multi-scale Analysis[J].Multimedia Tools and Applications,2017,76(5):7213-7234.
[17] ZHANG Z,XU Y,YANG J,et al.A Survey of Sparse Representation:Algorithms and Applications[J].IEEE Access,2017,3:490-530.
[18] ZHANG L,KANG B,LIU B,et al.Image Inpainting Based on Exemplar and Sparse Representation [J].International Journal of Signal Processing,Image Processingand Pattern Recognition,2016,9(9):177-188.
[19] AHARON M,ELAD M,BRUCKSTEIN A.K-SVD:An Algorithm for Designing Over-complete Dictionaries for Sparse Representation [J].IEEE Transactions on Signal Processing,2006,54(11):4311-4322.
[20] ELAD M,STARCK J L,QUERRE P,et al.Simultaneous Cartoon and Texture Image Inpainting Using Morphological Component Analysis(MCA)[J].Applied and Computational Harmonic Analysis,2005,19(3):340-358.
[21] DAI L,JIANG D H,DING B,et al.Improved Digital Image Restoration Algorithm Based on Criminisi[J].Journal of Digital Information Management,2016,14(5):302-310.
[22] ZHANG S H,WANG K G,ZHU X.Improved Criminisi Algorithm Constrained by Local Feature[J].Computer Engineering and Applications,2014,50(8):127-130.(in Chinese) 张申华,王克刚,祝轩.局部特征信息约束的改进Criminisi算法[J].计算机工程与应用,2014,50(8):127-130.
[23] LIU Y F,WANG F L,XI X Y.Enhanced Algorithm for Exemplar-based Image Inpainting [C]∥2013 9th International Conference on Computational Intelligence and Security.IEEE,2013:209-213.
[24] NAN A,XI X.An Improved CriminisiAlgorithm Based on ANew Priority Function and Updating Confidence[C]∥International Conference on Biomedical Engineering and Informatics.IEEE,2014:885-889.
[25] WANG J,LU K,PAN D,et al.Letters:Robust Object Removal with An Exemplar-based Image InpaintingApproach[J].Neurocomputing,2014,123:150-155.
[26] XI X,WANG F,LIU Y.Improved Criminisi Algorithm Based on a New Priority Function With the Gray Entropy [C]∥2013 9th International Conference on Computational Intelligence and Security.IEEE,2013:214-218.
[27] LIU Y F,WANG F L,XI X Y,et al.Improved Algorithm for Image Inpainting Based on Texture Synthesis [J].Journal of Chinese Computer Systems,2014,35(12):2754-2758.(in Chinese) 刘业妃,王福龙,奚祥艳,等.改进的Criminisi图像修复算法[J].小型微型计算机系统,2014,35(12):2754-2758.
[28] XU Z,SUN J.Image Inpainting by Patch Propagation Using Patch Sparsity [J].IEEE Transactions on Image Processing,2010,19(5):1153-1165.
[29] ARBELAEZ P,MAIRE M,FOWLKES C,et al.Contour Detection and Hierarchical Image Segmentation [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2011,33(5):898-916.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed