Computer Science ›› 2019, Vol. 46 ›› Issue (3): 119-124.doi: 10.11896/j.issn.1002-137X.2019.03.017

• ChinaMM2018 • Previous Articles     Next Articles

Deep Convolutional Prior Guided Robust Image Separation Method and Its Applications

JIANG Zhi-ying, LIU Ri-sheng   

  1. (Dalian University of Technology-Ritsumeikan University International School of Information Science & Engineering,Dalian,Liaoning 116621,China)
    (Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province,Dalian,Liaoning 116621,China)
  • Received:2018-07-22 Revised:2018-09-25 Online:2019-03-15 Published:2019-03-22

Abstract: Single image layer separation aims to divide the observed image into two independent and practical components based on the requirement of tasks.Many tasks in computer vision can be understood as the separation of two different layers essentially,such as single image rain streak removal,intrinsic image decomposition and reflection removal.Therefore,an excellent image layer decomposition method would promote the solution of these problems greatly.Since there is only one known variable,two variables need to be recovered.This problem is fundamentally ill-posed.Most exis-ting approaches tend to design complex priors according to the different characteristics between the two separated layers.However,loss function with complex prior regularization is hard to be optimized.Performance is also compromised by the fixed iteration schemes and less data fitting ability.More importantly,these conventional prior based methods can only be applied to one specific task as they are weak in generalization.To partially mitigate the limitations mentioned above,this paper developed a flexible optimization technique to incorporate deep architectures into optimization iterations for adaptive image layer separation.As we all know,the convolutional neural network is a network structure composed of convolutions and other non-linear operations.

第3期姜智颖,等:深度卷积先验引导的鲁棒图像层分离方法及其应用
The convolution operation uses different convolution kernels to extract different features for a given image,so the convolution kernel has very strong capabilities for feature extraction.Recently,the advantages of deep learning in feature extraction have been gradually reflected and are increasingly used in the low-level image processing.Therefore,the proposed method uses deep convolutional prior instead of traditional model prior to characterize different layers.At the same time,in order to reduce the network’s dependence on training data and improve the effectiveness of the algorithm on different tasks,deep information is combined with traditional optimization framework.Specifically,energy function using MAP (Maximum A Posteriori) is built and then the model is transfered to three subproblems based on ADMM (Alternating Direction Method of Multipliers).The first two subproblems are to estimate two approximate separated layers,and the other subproblem is to solve the final result.In other words,deep convolutional networks are used to guide the process of model optimization.In this way,the proposed method not only retains the advantage of feature extraction in deep structure,but also maintains the stability of traditional model optimization and improves the effectiveness of networks.Finally,this method is applied to a variety of ima-ge restoration tasks,including single image rain streak removal and reflection removal.By comparing this method with several tasks-specific methods including conventional model methods and deep learning methods respectively,this me-thod shows great advantages in both visual effects and numerical results.It reveals that this method has a strong genera-lization in multi-tasks and outperforms other methods in each task.

Key words: Layer separation, Maximum a posteriori estimate, Rain streak removal, Reflection removal, Residual network

CLC Number: 

  • TP391
[1]GU S H,MENG D Y,ZUO W M,et al.Joint Convolutional
Analysis and Synthesis Sparse Representation for Single Image Layer Separation[C]∥Proceedings of the IEEE International Conference on Computer Vision.Venice,Italy,2017:1717-1725.
[2]LIANG Z T,XU J,ZHANG D,et al.A Hybrid l1l0 Layer Decomposition Model for Tone Mapping[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Salt Lake City,USA,2018.
[3]KANG L W,LIN C W,FU Y H.Automatic single-image-based rain streaks removal via image decomposition[J].IEEE Transactions on Image Processing,2012,21(4):1742-1755.
[4]HUANG D A,KANG L W,YANG M C,et al.Context-Aware
Single Image Rain Removal[C]∥Proceedings of the IEEE International Conference on Multimedia and Expo.Melbourne,Australia,2012:164-169.
[5]LUO Y,XU Y,JI H.Removing rain from a single image via discriminative sparse coding[C]∥Proceedings of the IEEE International Conference on Computer Vision.Santiago,Chile,2015:3397-3405.
[6]LI Y,TAN R T,GUO X J,et al.Rain streak removal using layer priors[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas,USA,2016:2736-2744.
[7]FU X Y,HUANG J B,DING X H,et al.Clearing the Skies:A deep network architecture for single-image rain removal[J].IEEE Transactions on Image Processing,2017,26(6):2944-2956.
[8]FU X Y,HUANG J B,ZENG D L,et al.Removing Rain from Single Images via a Deep Detail Network[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Honolulu,Hawaii,2017:3855-3863.
[9]YANG W H,TAN R T,FENG J S,et al.DeepJoint Rain Detection and Removal from a Single Image[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Honolulu,Hawaii,2017:1357-1366.
[10]LI Y,BROWN M S.Single Image Layer Separation Using Relative Smoothness[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Columbus,USA,2014:2752-2759.
[11]SHIH Y C,KRISHNAN D,DURAND F,et al.Reflection re-
moval using ghosting cues[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Boston,USA,2015:3193-3201.
[12]SPRINGER O,WEISS Y.Reflection separation using guided annotation[C]∥Proceedings of the IEEE International Conference on Image Processing.Beijing,China,2017:1192-1196.
[13]ZHANG K,ZUO W M,GU S H,et al.Learning deep CNN denoiser prior for image restoration[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Honolulu,Hawaii,2017.
[14]BOYD S,PARIKH N,CHU E,et al.Distributed optimization
and statistical learning via the alternating direction method of multipliers[J].Foundations and Trends in Machine Learning,2011,3(1):1-122.
[15]ZHU C,BYRD R H,LU P,et al.Algorithm 778:L-BFGS-B:Fortran subroutines for large-scale bound-constrained optimization[J].ACM Transactions on Mathematical Software,1997,23(4):550-560.
[16]MARTIN D,FOWLKES C,TAL D,et al.A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]∥Proceedings of the IEEE International Conference on Computer Vision.Vancouver,Canada,2001:416-423.
[17]GARG K,NAYAR S K.Photorealistic rendering of rain streaks[J].ACM Transactions on Graphics,2006,25(3):996-1002.
[18]LI Y,BROWN M S.Exploiting reflection change for automatic reflection removal[C]∥Proceedings of the IEEE International Conference on Computer Vision.Sydney,Australia,2013:2432-2439.
[1] WANG Xin-tong, WANG Xuan, SUN Zhi-xin. Network Traffic Anomaly Detection Method Based on Multi-scale Memory Residual Network [J]. Computer Science, 2022, 49(8): 314-322.
[2] GAO Rong-hua, BAI Qiang, WANG Rong, WU Hua-rui, SUN Xiang. Multi-tree Network Multi-crop Early Disease Recognition Method Based on Improved Attention Mechanism [J]. Computer Science, 2022, 49(6A): 363-369.
[3] HAN Hong-qi, RAN Ya-xin, ZHANG Yun-liang, GUI Jie, GAO Xiong, YI Meng-lin. Study on Cross-media Information Retrieval Based on Common Subspace Classification Learning [J]. Computer Science, 2022, 49(5): 33-42.
[4] ZHAO Ren-xing, XU Pin-jie, LIU Yao. ECG-based Atrial Fibrillation Detection Based on Deep Convolutional Residual Neural Network [J]. Computer Science, 2022, 49(5): 186-193.
[5] QU Zhong, CHEN Wen. Concrete Pavement Crack Detection Based on Dilated Convolution and Multi-features Fusion [J]. Computer Science, 2022, 49(3): 192-196.
[6] GUO Lin, LI Chen, CHEN Chen, ZHAO Rui, FAN Shi-lin, XU Xing-yu. Image Super-resolution Reconstruction Using Recursive ResidualNetwork Based on ChannelAttention [J]. Computer Science, 2021, 48(8): 139-144.
[7] XU Hua-jie, ZHANG Chen-qiang, SU Guo-shao. Accurate Segmentation Method of Aerial Photography Buildings Based on Deep Convolutional Residual Network [J]. Computer Science, 2021, 48(8): 169-174.
[8] BAO Yu-xuan, LU Tian-liang, DU Yan-hui, SHI Da. Deepfake Videos Detection Method Based on i_ResNet34 Model and Data Augmentation [J]. Computer Science, 2021, 48(7): 77-85.
[9] NIU Kang-li, CHEN Yu-zhang, ZHANG Gong-ping, TAN Qian-cheng, WANG Yi-chong, LUO Mei-qi. Vehicle Flow Measuring of UVA Based on Deep Learning [J]. Computer Science, 2021, 48(6A): 275-280.
[10] WANG Jian-ming, LI Xiang-feng, YE Lei, ZUO Dun-wen, ZHANG Li-ping. Medical Image Deblur Using Generative Adversarial Networks with Channel Attention [J]. Computer Science, 2021, 48(6A): 101-106.
[11] CHAI Bing, LI Dong-dong, WANG Zhe, GAO Da-qi. EEG Emotion Recognition Based on Frequency and Channel Convolutional Attention [J]. Computer Science, 2021, 48(12): 312-318.
[12] LIU Zun-xiong, ZHU Cheng-jia, HUANG Ji, CAI Ti-jian. Image Super-resolution by Residual Attention Network with Multi-skip Connection [J]. Computer Science, 2021, 48(11): 258-267.
[13] WANG Run-zheng, GAO Jian, HUANG Shu-hua, TONG Xin. Malicious Code Family Detection Method Based on Knowledge Distillation [J]. Computer Science, 2021, 48(1): 280-286.
[14] SONG Ya-fei, CHEN Yu-zhang, SHEN Jun-feng and ZENG Zhang-fan. Underwater Image Reconstruction Based on Improved Residual Network [J]. Computer Science, 2020, 47(6A): 500-504.
[15] ZHU Wei, WANG Tu-qiang, CHEN Yue-feng, HE De-feng. Object-level Edge Detection Algorithm Based on Multi-scale Residual Network [J]. Computer Science, 2020, 47(6): 144-150.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!