Computer Science ›› 2020, Vol. 47 ›› Issue (2): 106-111.doi: 10.11896/jsjkx.190100228

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Single Image De-raining Method Based on Deep Adjacently Connected Networks

FU Xue-yang,SUN Qi,HUANG Yue,DING Xing-hao   

  1. (School of Information Science and Technology,Xiamen University,Xiamen,Fujian 361005,China)
  • Received:2019-01-28 Online:2020-02-15 Published:2020-03-18
  • About author:U Xue-yang,born in 1988,Ph.D,is member of China Computer Federation (CCF).His main research interests include image processing and machine learning;DING Xing-hao,born in 1976,Ph.D,professor,Ph.D supervisor,is member of China Computer Federation (CCF).His main research interests include computer vision,machine learning,big data analysis and processing,sparse representation theory and artificial intelligence.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China (61571382, 81671766, 61571005, 81671674, 61671309, U1605252).

Abstract: Rain streaks result in the occlusion of image content,which seriously affects the human visual effect and the processing performance of subsequent systems.Existing deep learning-based methods improve de-raining performance at the expense of complex network structure and parameter burden,which makes these methods difficult for serving practical applications.Therefore,a deep adjacently connected network structure was proposedin this paper.By focusing on the relationship between learned feature maps in depth networks,a fusion operation is designed to connect the adjacent features to obtain rich and more effective feature representation.Experiments on three public synthetic datasets and real-world rainy images show that the proposed method improves de-raining performance on both subjective and objective evaluations.The average structural similarity (SSIM) value on the synthetic dataset Rain100H is 0.84,and the average SSIM values on the synthetic dataset Rain100L and Rain1200 are 0.96 and 0.91.In real-world rainy images,the proposed method can effectively remove the foreground rain streaks while protecting background image information to obtain better visual quality.Compared with JORDER,the proposed method achieves comparable de-raining results,and can reduce the model parameters and CPU runtime by one and two orders of magnitude,respectively.Experimental data demonstrate that fusing adjacent features in the deep network can generate more effective representation.Therefore,although the proposed method contains relative few parameters and simple neural network structure,it can still achieve better ima-ge de-raining performance and solve the problems of parameter burden and complex network structure in existing methods.Mo-reover,the network structure design scheme in this paper can also provide reference values for relative image restoration tasks based on deep learning.

Key words: Image de-raining, Deep learning, Convolutional neural networks, Feature fusion

CLC Number: 

  • TP391.41
[1]KANG L W,LIN C W,FU Y H.Automatic Single-Image-Based Rain StreaksRemoval via Image Decomposition[J].IEEE Tran-sactions on Image Processing,2011,21(4):1742-1755.
[2]HUANG D A,KANG L W,WANG Y C F,et al.Self-Learning BasedImage Decomposition with Applications to Single Image Denoising[J].IEEE Transactionsons on Multimedia,2014,16(1):83-93.
[3]CHEN Y L,HSU C T.A Generalized Low-Rank Appearance Model for Spatio-Temporally Correlated Rain Streaks[C]∥Processings of the the 2013 IEEE International Conference on Computer Vision.Sydney,Washington:IEEE,2013:1968-1975.
[4]WANG Y L,LIU S C,CHEN C,et al.A Hierarchical Approach for Rainor Snow Removing in a Single Color Image[J].IEEE Transactions on Image Processing,2017,26(8):3936-3950.
[5]LUO Y,XU Y,JI H.Removing Rain from a Single Image via Discriminative Sparse Coding[C]∥2015 IEEE International Conference on Computer Vision.Chile:Santiago,2015:3397-3405.
[6]LI Y,TAN R T,GUO X J,et al.Rain Streak Removal Using Layer Priors[C]∥2016IEEE Conference on Computer Vision and Pattern Recognition(CPVR).Las Vegas,USA,2016:2736-2744.
[7]ZHU L,FU C,LISCHINSKI D,et al.Joint Bi-layer Optimization for Single-image Rain Streak Removal[C]∥The IEEE International Conference on Computer Vision.Venice,Italy,2017:2545-2553.
[8]CHANG Y,YAN L,ZHONG S.Transformed Low-Rank Model for Line Pattern NoiseRemoval[C]∥2017 IEEE International Conference on Computer Vision.Venice,Italy,2017:1735-1743.
[9]GU S H,MENG D Y,ZUO W M,et al.Joint Convolutional Analysis andSynthesis Sparse Representation for Single Image Layer Separation[C]∥2017 IEEE InternationalConference on Computer Vision.Venice,Italy,2017:1717-1725.
[10]KRIZHEVSKY A,SUTSKEVER I,HINTON G.ImageNet classification with deep convolutional neural networks[C]∥Proceedings of the 25th International Conference on Neural Information Processing Systems.Lake Tahoe,Nevada,2012:1097-1105.
[11]EIGEN D,KRISHNAN D,FERGUS R.Restoring an Image Taken through a Window Covered with Dirt or Rain[C]∥Proceedings of the 2013 IEEE International Conference on Computer Vision.Washington,USA,2013:633-640.
[12]FU X Y,HUANG J B,DING X H,et al.Clearing the skies:a deep network architecture for single-image rain removal[J]. IEEE Transactions on Image Processing,2017,26(6):2944-2956.
[13]FU X Y,HUANG J B,ZENG D L,et al.Removing rain from single images via a deep detail network[C]∥ IEEE Conference on Computer Vision and Pattern Recognition.Honolulu,USA,2017:3855-3863.
[14]YANG W H,TAN R T,FENG J S,et al.Deep Joint Rain Detection and Removal from a Single Image[C]∥2017 IEEE Conference on ComputerVision and Pattern Recognition.Honolulu,USA,2017:1685-1694.
[15]PAN J S,LIU S F,ZHANG J W,et al.Learning Dual Convolutional Neural Networksfor Low-Level Vision[C]∥IEEE Conference on Computer Vision and Pattern Recognition.Salt Lake City,USA,2018:1-10.
[16]ZHANG H,PATEL V M.Density-Aware Single Image De-raining using a Multi-stream Dense Network[C]∥IEEE Confe-rence on Computer Vision and Pattern Recognition.Salt Lake City,USA,2018:1-10.
[17]YU F,KOLTUN V,FUNKHOUSER T.Dilated residual net-works[C]∥IEEE Conference on Computer Vision and Pattern Recognition.Honolulu,USA,2017:636-644.
[18]HE K M,ZHANG X Y,REN S Q,et al.Deep residual learning for image recognition[C]∥IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas,United States,2016:770-778.
[19]HUANG G,LIU Z,MAATEN L,et al.Densely connected convolutional networks[C]∥IEEE Conference on Computer Vision and Pattern Recognition.Honolulu,USA,2017:4700-4708.
[20]KINGMA D,BA J.Adam:A Method for Stochastic Optimization [C]∥International Conference on Learning Representations.San Diego,USA,2015:1-15.
[21]WANG Z,BOVIK A C,SHEIKH H R,et al.Image Quality Assessment:from Error Visibility to Structural Similarity[J].IEEE Transactions on Image Processing,2004,13(4):600-612.
[22]GONZALEZ R C,WOODS R E.Digital Image Processing[M].Upper Saddle River:Prentice Hall,2012.
[1] WANG Rui-ping, JIA Zhen, LIU Chang, CHEN Ze-wei, LI Tian-rui. Deep Interest Factorization Machine Network Based on DeepFM [J]. Computer Science, 2021, 48(1): 226-232.
[2] ZHANG Fan, HE Wen-qi, JI Hong-bing, LI Dan-ping, WANG Lei. Multi-view Dictionary-pair Learning Based on Block-diagonal Representation [J]. Computer Science, 2021, 48(1): 233-240.
[3] YU Wen-jia, DING Shi-fei. Conditional Generative Adversarial Network Based on Self-attention Mechanism [J]. Computer Science, 2021, 48(1): 241-246.
[4] TONG Xin, WANG Bin-jun, WANG Run-zheng, PAN Xiao-qin. Survey on Adversarial Sample of Deep Learning Towards Natural Language Processing [J]. Computer Science, 2021, 48(1): 258-267.
[5] DING Yu, WEI Hao, PAN Zhi-song, LIU Xin. Survey of Network Representation Learning [J]. Computer Science, 2020, 47(9): 52-59.
[6] HE Xin, XU Juan, JIN Ying-ying. Action-related Network:Towards Modeling Complete Changeable Action [J]. Computer Science, 2020, 47(9): 123-128.
[7] YE Ya-nan, CHI Jing, YU Zhi-ping, ZHAN Yu-liand ZHANG Cai-ming. Expression Animation Synthesis Based on Improved CycleGan Model and Region Segmentation [J]. Computer Science, 2020, 47(9): 142-149.
[8] DENG Liang, XU Geng-lin, LI Meng-jie, CHEN Zhang-jin. Fast Face Recognition Based on Deep Learning and Multiple Hash Similarity Weighting [J]. Computer Science, 2020, 47(9): 163-168.
[9] BAO Yu-xuan, LU Tian-liang, DU Yan-hui. Overview of Deepfake Video Detection Technology [J]. Computer Science, 2020, 47(9): 283-292.
[10] SUN Yan-li, YE Jiong-yao. Convolutional Neural Networks Compression Based on Pruning and Quantization [J]. Computer Science, 2020, 47(8): 261-266.
[11] YUAN Ye, HE Xiao-ge, ZHU Ding-kun, WANG Fu-lee, XIE Hao-ran, WANG Jun, WEI Ming-qiang, GUO Yan-wen. Survey of Visual Image Saliency Detection [J]. Computer Science, 2020, 47(7): 84-91.
[12] WANG Wen-dao, WANG Run-ze, WEI Xin-lei, QI Yun-liang, MA Yi-de. Automatic Recognition of ECG Based on Stacked Bidirectional LSTM [J]. Computer Science, 2020, 47(7): 118-124.
[13] LIU Yan, WEN Jing. Complex Scene Text Detection Based on Attention Mechanism [J]. Computer Science, 2020, 47(7): 135-140.
[14] ZHANG Zhi-yang, ZHANG Feng-li, TAN Qi, WANG Rui-jin. Review of Information Cascade Prediction Methods Based on Deep Learning [J]. Computer Science, 2020, 47(7): 141-153.
[15] JIANG Wen-bin, FU Zhi, PENG Jing, ZHU Jian. 4Bit-based Gradient Compression Method for Distributed Deep Learning System [J]. Computer Science, 2020, 47(7): 220-226.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] LEI Li-hui and WANG Jing. Parallelization of LTL Model Checking Based on Possibility Measure[J]. Computer Science, 2018, 45(4): 71 -75 .
[2] SUN Qi, JIN Yan, HE Kun and XU Ling-xuan. Hybrid Evolutionary Algorithm for Solving Mixed Capacitated General Routing Problem[J]. Computer Science, 2018, 45(4): 76 -82 .
[3] ZHANG Jia-nan and XIAO Ming-yu. Approximation Algorithm for Weighted Mixed Domination Problem[J]. Computer Science, 2018, 45(4): 83 -88 .
[4] WU Jian-hui, HUANG Zhong-xiang, LI Wu, WU Jian-hui, PENG Xin and ZHANG Sheng. Robustness Optimization of Sequence Decision in Urban Road Construction[J]. Computer Science, 2018, 45(4): 89 -93 .
[5] SHI Wen-jun, WU Ji-gang and LUO Yu-chun. Fast and Efficient Scheduling Algorithms for Mobile Cloud Offloading[J]. Computer Science, 2018, 45(4): 94 -99 .
[6] ZHOU Yan-ping and YE Qiao-lin. L1-norm Distance Based Least Squares Twin Support Vector Machine[J]. Computer Science, 2018, 45(4): 100 -105 .
[7] LIU Bo-yi, TANG Xiang-yan and CHENG Jie-ren. Recognition Method for Corn Borer Based on Templates Matching in Muliple Growth Periods[J]. Computer Science, 2018, 45(4): 106 -111 .
[8] GENG Hai-jun, SHI Xin-gang, WANG Zhi-liang, YIN Xia and YIN Shao-ping. Energy-efficient Intra-domain Routing Algorithm Based on Directed Acyclic Graph[J]. Computer Science, 2018, 45(4): 112 -116 .
[9] CUI Qiong, LI Jian-hua, WANG Hong and NAN Ming-li. Resilience Analysis Model of Networked Command Information System Based on Node Repairability[J]. Computer Science, 2018, 45(4): 117 -121 .
[10] WANG Zhen-chao, HOU Huan-huan and LIAN Rui. Path Optimization Scheme for Restraining Degree of Disorder in CMT[J]. Computer Science, 2018, 45(4): 122 -125 .