Computer Science ›› 2020, Vol. 47 ›› Issue (2): 106-111.doi: 10.11896/jsjkx.190100228

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Single Image De-raining Method Based on Deep Adjacently Connected Networks

FU Xue-yang,SUN Qi,HUANG Yue,DING Xing-hao   

  1. (School of Information Science and Technology,Xiamen University,Xiamen,Fujian 361005,China)
  • Received:2019-01-28 Online:2020-02-15 Published:2020-03-18
  • About author:U Xue-yang,born in 1988,Ph.D,is member of China Computer Federation (CCF).His main research interests include image processing and machine learning;DING Xing-hao,born in 1976,Ph.D,professor,Ph.D supervisor,is member of China Computer Federation (CCF).His main research interests include computer vision,machine learning,big data analysis and processing,sparse representation theory and artificial intelligence.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China (61571382, 81671766, 61571005, 81671674, 61671309, U1605252).

Abstract: Rain streaks result in the occlusion of image content,which seriously affects the human visual effect and the processing performance of subsequent systems.Existing deep learning-based methods improve de-raining performance at the expense of complex network structure and parameter burden,which makes these methods difficult for serving practical applications.Therefore,a deep adjacently connected network structure was proposedin this paper.By focusing on the relationship between learned feature maps in depth networks,a fusion operation is designed to connect the adjacent features to obtain rich and more effective feature representation.Experiments on three public synthetic datasets and real-world rainy images show that the proposed method improves de-raining performance on both subjective and objective evaluations.The average structural similarity (SSIM) value on the synthetic dataset Rain100H is 0.84,and the average SSIM values on the synthetic dataset Rain100L and Rain1200 are 0.96 and 0.91.In real-world rainy images,the proposed method can effectively remove the foreground rain streaks while protecting background image information to obtain better visual quality.Compared with JORDER,the proposed method achieves comparable de-raining results,and can reduce the model parameters and CPU runtime by one and two orders of magnitude,respectively.Experimental data demonstrate that fusing adjacent features in the deep network can generate more effective representation.Therefore,although the proposed method contains relative few parameters and simple neural network structure,it can still achieve better ima-ge de-raining performance and solve the problems of parameter burden and complex network structure in existing methods.Mo-reover,the network structure design scheme in this paper can also provide reference values for relative image restoration tasks based on deep learning.

Key words: Convolutional neural networks, Deep learning, Feature fusion, Image de-raining

CLC Number: 

  • TP391.41
[1]KANG L W,LIN C W,FU Y H.Automatic Single-Image-Based Rain StreaksRemoval via Image Decomposition[J].IEEE Tran-sactions on Image Processing,2011,21(4):1742-1755.
[2]HUANG D A,KANG L W,WANG Y C F,et al.Self-Learning BasedImage Decomposition with Applications to Single Image Denoising[J].IEEE Transactionsons on Multimedia,2014,16(1):83-93.
[3]CHEN Y L,HSU C T.A Generalized Low-Rank Appearance Model for Spatio-Temporally Correlated Rain Streaks[C]∥Processings of the the 2013 IEEE International Conference on Computer Vision.Sydney,Washington:IEEE,2013:1968-1975.
[4]WANG Y L,LIU S C,CHEN C,et al.A Hierarchical Approach for Rainor Snow Removing in a Single Color Image[J].IEEE Transactions on Image Processing,2017,26(8):3936-3950.
[5]LUO Y,XU Y,JI H.Removing Rain from a Single Image via Discriminative Sparse Coding[C]∥2015 IEEE International Conference on Computer Vision.Chile:Santiago,2015:3397-3405.
[6]LI Y,TAN R T,GUO X J,et al.Rain Streak Removal Using Layer Priors[C]∥2016IEEE Conference on Computer Vision and Pattern Recognition(CPVR).Las Vegas,USA,2016:2736-2744.
[7]ZHU L,FU C,LISCHINSKI D,et al.Joint Bi-layer Optimization for Single-image Rain Streak Removal[C]∥The IEEE International Conference on Computer Vision.Venice,Italy,2017:2545-2553.
[8]CHANG Y,YAN L,ZHONG S.Transformed Low-Rank Model for Line Pattern NoiseRemoval[C]∥2017 IEEE International Conference on Computer Vision.Venice,Italy,2017:1735-1743.
[9]GU S H,MENG D Y,ZUO W M,et al.Joint Convolutional Analysis andSynthesis Sparse Representation for Single Image Layer Separation[C]∥2017 IEEE InternationalConference on Computer Vision.Venice,Italy,2017:1717-1725.
[10]KRIZHEVSKY A,SUTSKEVER I,HINTON G.ImageNet classification with deep convolutional neural networks[C]∥Proceedings of the 25th International Conference on Neural Information Processing Systems.Lake Tahoe,Nevada,2012:1097-1105.
[11]EIGEN D,KRISHNAN D,FERGUS R.Restoring an Image Taken through a Window Covered with Dirt or Rain[C]∥Proceedings of the 2013 IEEE International Conference on Computer Vision.Washington,USA,2013:633-640.
[12]FU X Y,HUANG J B,DING X H,et al.Clearing the skies:a deep network architecture for single-image rain removal[J]. IEEE Transactions on Image Processing,2017,26(6):2944-2956.
[13]FU X Y,HUANG J B,ZENG D L,et al.Removing rain from single images via a deep detail network[C]∥ IEEE Conference on Computer Vision and Pattern Recognition.Honolulu,USA,2017:3855-3863.
[14]YANG W H,TAN R T,FENG J S,et al.Deep Joint Rain Detection and Removal from a Single Image[C]∥2017 IEEE Conference on ComputerVision and Pattern Recognition.Honolulu,USA,2017:1685-1694.
[15]PAN J S,LIU S F,ZHANG J W,et al.Learning Dual Convolutional Neural Networksfor Low-Level Vision[C]∥IEEE Conference on Computer Vision and Pattern Recognition.Salt Lake City,USA,2018:1-10.
[16]ZHANG H,PATEL V M.Density-Aware Single Image De-raining using a Multi-stream Dense Network[C]∥IEEE Confe-rence on Computer Vision and Pattern Recognition.Salt Lake City,USA,2018:1-10.
[17]YU F,KOLTUN V,FUNKHOUSER T.Dilated residual net-works[C]∥IEEE Conference on Computer Vision and Pattern Recognition.Honolulu,USA,2017:636-644.
[18]HE K M,ZHANG X Y,REN S Q,et al.Deep residual learning for image recognition[C]∥IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas,United States,2016:770-778.
[19]HUANG G,LIU Z,MAATEN L,et al.Densely connected convolutional networks[C]∥IEEE Conference on Computer Vision and Pattern Recognition.Honolulu,USA,2017:4700-4708.
[20]KINGMA D,BA J.Adam:A Method for Stochastic Optimization [C]∥International Conference on Learning Representations.San Diego,USA,2015:1-15.
[21]WANG Z,BOVIK A C,SHEIKH H R,et al.Image Quality Assessment:from Error Visibility to Structural Similarity[J].IEEE Transactions on Image Processing,2004,13(4):600-612.
[22]GONZALEZ R C,WOODS R E.Digital Image Processing[M].Upper Saddle River:Prentice Hall,2012.
[1] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[2] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[3] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[4] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[5] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[6] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[7] ZHU Cheng-zhang, HUANG Jia-er, XIAO Ya-long, WANG Han, ZOU Bei-ji. Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism [J]. Computer Science, 2022, 49(8): 113-119.
[8] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[9] HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[10] ZHANG Ying-tao, ZHANG Jie, ZHANG Rui, ZHANG Wen-qiang. Photorealistic Style Transfer Guided by Global Information [J]. Computer Science, 2022, 49(7): 100-105.
[11] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[12] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[13] ZHOU Hui, SHI Hao-chen, TU Yao-feng, HUANG Sheng-jun. Robust Deep Neural Network Learning Based on Active Sampling [J]. Computer Science, 2022, 49(7): 164-169.
[14] SU Dan-ning, CAO Gui-tao, WANG Yan-nan, WANG Hong, REN He. Survey of Deep Learning for Radar Emitter Identification Based on Small Sample [J]. Computer Science, 2022, 49(7): 226-235.
[15] LIU Wei-ye, LU Hui-min, LI Yu-peng, MA Ning. Survey on Finger Vein Recognition Research [J]. Computer Science, 2022, 49(6A): 1-11.
Full text



No Suggested Reading articles found!