Computer Science

Computer Science ›› 2024, Vol. 51 ›› Issue (1): 190-197.doi: 10.11896/jsjkx.230500125

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Raindrop In-Situ Captured Benchmark Image Dataset and Evaluation

CHEN Tianyi1, XUE Wen1, QUAN Yuhui1, XU Yong1,2   

  1. 1 School of Computer Science and Engineering,South China University of Technology,Guangzhou 510006,China
    2 Peng Cheng Laboratory,Shenzhen,Guangdong 518055,China
  • Received:2023-05-21 Revised:2023-09-28 Online:2024-01-15 Published:2024-01-12
  • About author:CHEN Tianyi,born in 1997,Ph.D candidate,is a student member of CCF(No.H5052G).His main research interests include computer vision,image proces-sing and image generation.
    QUAN Yuhui,born in 1985,Ph.D,associate professor,is a member of CCF(No.70281S).His main research in-terests include image processing,deep learning and sparse representation.
  • Supported by:
    National Natural Science Foundation of China(62072188).

PDF (PC)

1841

Abstract: When taking photos through glass windows in rainy days,the raindrops adhered to glass surfaces are usually presented in the images,which not only degrade the visibility of the image but also prevent many computer vision algorithms from functioning properly.The research on raindrop removal is a scientific research to remove raindrops from such rainy images.The singleimage raindrop removal research presents significant challenges due to the diverse and unique forms of raindrops found in nature.The varying transparency of raindrops further complicates the task of removing raindrop artifacts and degrades the imaging quality of background scenes,adversely impacting the performance of existing raindrop removal algorithms.To facilitate a comprehensive understanding of this research area,this paper provides a detailed introduction to single-image raindrop removal,covering two main aspects:single-image raindrop removal algorithms and joint raindrop removal algorithms for single images.Additionally,a summary and evaluation of existing algorithms in this field are presented.However,the performance of the algorithm is often li-mited by the quality and quantity of the dataset in deep learning based methods,but in existing raindrop datasets,common situations such as low-quality raindrop images and insufficient image quantities exist.In existing raindrop datasets,there are common situations such as poor quality of raindrop images and insufficient number of raindrop images.This paper proposes a higher education megacenter(HEMC) dataset.Camera shake,window reflections and other external disturbances are avoided as much as possible thus improving the image quality of the training set and accuracy of the test set and indirectly improving the performance of the raindrop removal methods.HEMC is evaluated in various aspects using competent visual effects and objective metrics.Experimental results show the diversity of the raindrop images in HEMC and stability of the objective metrics.In addition,the results verify the universality and stability of the HEMC in the raindrop removal methods.

Key words: Image raindrop removal, Raindrop image dataset, Deep learning, Image evaluation metric

CLC Number: 

  • TP391.4
[1]YOU S,TAN R,KAWAKAMI R,et al.Adherent raindrop detection and removal in video[C]//Proceedings of the IEEE Conference Computer Vision Pattern Recognition.Portland,USA,2013:1035-1042.
[2]QIAN R,TAN R,YANG W,et al.Attentive generative adversarial network for raindrop removal from a single image[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Salt Lake City,USA,2018:2482-2491.
[3]QUAN Y,DENG S,CHEN Y,et al.Deep learning for seeingthrough window with raindrops[C]//Proceedings of the IEEE International Conference Computer Vision.Seoul,Korea(South),2019:2463-2471.
[4]QUAN R,YU X,LIANG Y,et al.Removing raindrops and rain streaks in one go[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Virtual.2021:9147-9156.
[5]MIAO Y,HONG H,KIM H.Size and angle filter based rain removal in video for outdoor surveillance systems[C]//IEEE Asian Control Conference.Taipei,2011:1300-1304.
[6]EIGEN D,KRISHNAN D,FERGUS R,et al.Restoring animage taken through a window covered with dirt or rain[C]//Proceedings of the IEEE International Conference on Computer Vision.Sydney,Australia,2013:633-640.
[7]ISOLA P,ZHU J Y,ZHOU T,et al.Image-to-image translation with conditional adversarial networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Honolulu,Hawaii,2017:1125-1134.
[8]PENG J,XU Y,CHEN T,et al.Single-image raindrop removal using concurrent channel-spatial attention and long-short skip connections[J].Pattern Recognition Letters,2020,131:121-127.
[9]HE D,SHANG X,LUO J,et al.Adherent mist and raindrop removal from a single image using attentive convolutional network[J].Neurocomputing,2022,505:178-187.
[10]WEI Y,ZHANG Z,XU M,et al.Robust attention deraining network for synchronous rain streaks and raindrops removal[C]//Proceedings of the 30th ACM International Conference on Multimedia.Lisbon,Portugal,2022:6464-6472.
[11]RONNEBERGER O,FISCHER P,BROX T,et al.U-net:Con-volutional networks for biomedical image segmentation[C]//Proceedings of the International Conference on MedicalImage Computing and Computer-assisted Intervention.Munich,Germany,2015:234-241.
[12]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas,USA,2016:770-778.
[13]HAO Z,YOU S,LI Y,et al.Learning from synthetic photorea-listic raindrop for single image raindrop removal[C]//Procee-dings of the IEEE International Conference on Computer Vision Workshops.Seoul,Korea(South),2019.
[14]ALLETTO S,CARLIN C,RIGAZIO L,et al.Adherent raindrop removal with self-supervised attention maps and spatio-temporal generative adversarial networks[C]//Proceedings of the IEEEInternational Conference on Computer Vision Workshops.Seoul,Korea(South),2019.
[15]REN D,LI J,HAN M,et al.Not All Areas Are Equal:A Novel Separation-Restoration-Fusion Network for Image Raindrop Removal[J].Computer Graphics Forum,2020,39(7):495-505.
[16]LUO W,LAI J,XIE X,et al.Weakly supervised learning for raindrop removal on a single image[J].IEEE Transactions on Circuits and Systems for Video Technology,2020,31(5):1673-1683.
[17]SHAO M W,LI L,MENG D Y,et al.Uncertainty guided multi-scale attention network for raindrop removal from a single image[J].IEEE Transactions on Image Processing,2021,30:4828-4839.
[18]SHAO M,LI L,WANG H,et al.Selective generative adversarial network for raindrop removal from a single image[J].Neurocomputing,2021,426:265-273.
[19]YAN X,LOKE Y R.RainGAN:Unsupervised Raindrop Remo-val via Decomposition and Composition[C]//Proceedings of the IEEE Winter Conference on Applications of Computer Vision.Waikoloa,USA,2022:14-23.
[20]LIU X,SUGANUMA M,SUN Z,et al.Dual residual networks leveraging the potential of paired operations for image restoration[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Long Beach,California,USA,2019:7007-7016.
[21]GUO Y,CHEN J,REN X,et al.Joint raindrop and haze removal from a single image[J].IEEE Transactions on Image Proces-sing,2020,29:9508-9519.
[22]ZHANG K,LI D,LUO W,et al.Dual attention-in-attentionmodel for joint rain streak and raindrop removal[J].IEEE Transactions on Image Processing,2021,30:7608-7619.
[1] DU Yu, YU Zishu, PENG Xiaohui, XU Zhiwei. Padding Load:Load Reducing Cluster Resource Waste and Deep Learning Training Costs [J]. Computer Science, 2024, 51(9): 71-79.
[2] XU Jinlong, GUI Zhonghua, LI Jia'nan, LI Yingying, HAN Lin. FP8 Quantization and Inference Memory Optimization Based on MLIR [J]. Computer Science, 2024, 51(9): 112-120.
[3] SUN Yumo, LI Xinhang, ZHAO Wenjie, ZHU Li, LIANG Ya’nan. Driving Towards Intelligent Future:The Application of Deep Learning in Rail Transit Innovation [J]. Computer Science, 2024, 51(8): 1-10.
[4] KONG Lingchao, LIU Guozhu. Review of Outlier Detection Algorithms [J]. Computer Science, 2024, 51(8): 20-33.
[5] TANG Ruiqi, XIAO Ting, CHI Ziqiu, WANG Zhe. Few-shot Image Classification Based on Pseudo-label Dependence Enhancement and NoiseInterferenceReduction [J]. Computer Science, 2024, 51(8): 152-159.
[6] XIAO Xiao, BAI Zhengyao, LI Zekai, LIU Xuheng, DU Jiajin. Parallel Multi-scale with Attention Mechanism for Point Cloud Upsampling [J]. Computer Science, 2024, 51(8): 183-191.
[7] ZHANG Junsan, CHENG Ming, SHEN Xiuxuan, LIU Yuxue, WANG Leiquan. Diversified Label Matrix Based Medical Image Report Generation [J]. Computer Science, 2024, 51(8): 200-208.
[8] GUO Fangyuan, JI Genlin. Video Anomaly Detection Method Based on Dual Discriminators and Pseudo Video Generation [J]. Computer Science, 2024, 51(8): 217-223.
[9] CHEN Siyu, MA Hailong, ZHANG Jianhui. Encrypted Traffic Classification of CNN and BiGRU Based on Self-attention [J]. Computer Science, 2024, 51(8): 396-402.
[10] YANG Heng, LIU Qinrang, FAN Wang, PEI Xue, WEI Shuai, WANG Xuan. Study on Deep Learning Automatic Scheduling Optimization Based on Feature Importance [J]. Computer Science, 2024, 51(7): 22-28.
[11] LI Jiaying, LIANG Yudong, LI Shaoji, ZHANG Kunpeng, ZHANG Chao. Study on Algorithm of Depth Image Super-resolution Guided by High-frequency Information ofColor Images [J]. Computer Science, 2024, 51(7): 197-205.
[12] SHI Dianxi, GAO Yunqi, SONG Linna, LIU Zhe, ZHOU Chenlei, CHEN Ying. Deep-Init:Non Joint Initialization Method for Visual Inertial Odometry Based on Deep Learning [J]. Computer Science, 2024, 51(7): 327-336.
[13] FAN Yi, HU Tao, YI Peng. Host Anomaly Detection Framework Based on Multifaceted Information Fusion of SemanticFeatures for System Calls [J]. Computer Science, 2024, 51(7): 380-388.
[14] GAN Run, WEI Xianglin, WANG Chao, WANG Bin, WANG Min, FAN Jianhua. Backdoor Attack Method in Autoencoder End-to-End Communication System [J]. Computer Science, 2024, 51(7): 413-421.
[15] HUANG Haixin, CAI Mingqi, WANG Yuyao. Review of Point Cloud Semantic Segmentation Based on Graph Convolutional Neural Networks [J]. Computer Science, 2024, 51(6A): 230400196-7.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.