Computer Science

Computer Science ›› 2024, Vol. 51 ›› Issue (1): 190-197.doi: 10.11896/jsjkx.230500125

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Raindrop In-Situ Captured Benchmark Image Dataset and Evaluation

CHEN Tianyi1, XUE Wen1, QUAN Yuhui1, XU Yong1,2   

  1. 1 School of Computer Science and Engineering,South China University of Technology,Guangzhou 510006,China
    2 Peng Cheng Laboratory,Shenzhen,Guangdong 518055,China
  • Received:2023-05-21 Revised:2023-09-28 Online:2024-01-15 Published:2024-01-12
  • About author:CHEN Tianyi,born in 1997,Ph.D candidate,is a student member of CCF(No.H5052G).His main research interests include computer vision,image proces-sing and image generation.
    QUAN Yuhui,born in 1985,Ph.D,associate professor,is a member of CCF(No.70281S).His main research in-terests include image processing,deep learning and sparse representation.
  • Supported by:
    National Natural Science Foundation of China(62072188).

PDF (PC)

1518

Abstract: When taking photos through glass windows in rainy days,the raindrops adhered to glass surfaces are usually presented in the images,which not only degrade the visibility of the image but also prevent many computer vision algorithms from functioning properly.The research on raindrop removal is a scientific research to remove raindrops from such rainy images.The singleimage raindrop removal research presents significant challenges due to the diverse and unique forms of raindrops found in nature.The varying transparency of raindrops further complicates the task of removing raindrop artifacts and degrades the imaging quality of background scenes,adversely impacting the performance of existing raindrop removal algorithms.To facilitate a comprehensive understanding of this research area,this paper provides a detailed introduction to single-image raindrop removal,covering two main aspects:single-image raindrop removal algorithms and joint raindrop removal algorithms for single images.Additionally,a summary and evaluation of existing algorithms in this field are presented.However,the performance of the algorithm is often li-mited by the quality and quantity of the dataset in deep learning based methods,but in existing raindrop datasets,common situations such as low-quality raindrop images and insufficient image quantities exist.In existing raindrop datasets,there are common situations such as poor quality of raindrop images and insufficient number of raindrop images.This paper proposes a higher education megacenter(HEMC) dataset.Camera shake,window reflections and other external disturbances are avoided as much as possible thus improving the image quality of the training set and accuracy of the test set and indirectly improving the performance of the raindrop removal methods.HEMC is evaluated in various aspects using competent visual effects and objective metrics.Experimental results show the diversity of the raindrop images in HEMC and stability of the objective metrics.In addition,the results verify the universality and stability of the HEMC in the raindrop removal methods.

Key words: Image raindrop removal, Raindrop image dataset, Deep learning, Image evaluation metric

CLC Number: 

  • TP391.4
[1]YOU S,TAN R,KAWAKAMI R,et al.Adherent raindrop detection and removal in video[C]//Proceedings of the IEEE Conference Computer Vision Pattern Recognition.Portland,USA,2013:1035-1042.
[2]QIAN R,TAN R,YANG W,et al.Attentive generative adversarial network for raindrop removal from a single image[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Salt Lake City,USA,2018:2482-2491.
[3]QUAN Y,DENG S,CHEN Y,et al.Deep learning for seeingthrough window with raindrops[C]//Proceedings of the IEEE International Conference Computer Vision.Seoul,Korea(South),2019:2463-2471.
[4]QUAN R,YU X,LIANG Y,et al.Removing raindrops and rain streaks in one go[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Virtual.2021:9147-9156.
[5]MIAO Y,HONG H,KIM H.Size and angle filter based rain removal in video for outdoor surveillance systems[C]//IEEE Asian Control Conference.Taipei,2011:1300-1304.
[6]EIGEN D,KRISHNAN D,FERGUS R,et al.Restoring animage taken through a window covered with dirt or rain[C]//Proceedings of the IEEE International Conference on Computer Vision.Sydney,Australia,2013:633-640.
[7]ISOLA P,ZHU J Y,ZHOU T,et al.Image-to-image translation with conditional adversarial networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Honolulu,Hawaii,2017:1125-1134.
[8]PENG J,XU Y,CHEN T,et al.Single-image raindrop removal using concurrent channel-spatial attention and long-short skip connections[J].Pattern Recognition Letters,2020,131:121-127.
[9]HE D,SHANG X,LUO J,et al.Adherent mist and raindrop removal from a single image using attentive convolutional network[J].Neurocomputing,2022,505:178-187.
[10]WEI Y,ZHANG Z,XU M,et al.Robust attention deraining network for synchronous rain streaks and raindrops removal[C]//Proceedings of the 30th ACM International Conference on Multimedia.Lisbon,Portugal,2022:6464-6472.
[11]RONNEBERGER O,FISCHER P,BROX T,et al.U-net:Con-volutional networks for biomedical image segmentation[C]//Proceedings of the International Conference on MedicalImage Computing and Computer-assisted Intervention.Munich,Germany,2015:234-241.
[12]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas,USA,2016:770-778.
[13]HAO Z,YOU S,LI Y,et al.Learning from synthetic photorea-listic raindrop for single image raindrop removal[C]//Procee-dings of the IEEE International Conference on Computer Vision Workshops.Seoul,Korea(South),2019.
[14]ALLETTO S,CARLIN C,RIGAZIO L,et al.Adherent raindrop removal with self-supervised attention maps and spatio-temporal generative adversarial networks[C]//Proceedings of the IEEEInternational Conference on Computer Vision Workshops.Seoul,Korea(South),2019.
[15]REN D,LI J,HAN M,et al.Not All Areas Are Equal:A Novel Separation-Restoration-Fusion Network for Image Raindrop Removal[J].Computer Graphics Forum,2020,39(7):495-505.
[16]LUO W,LAI J,XIE X,et al.Weakly supervised learning for raindrop removal on a single image[J].IEEE Transactions on Circuits and Systems for Video Technology,2020,31(5):1673-1683.
[17]SHAO M W,LI L,MENG D Y,et al.Uncertainty guided multi-scale attention network for raindrop removal from a single image[J].IEEE Transactions on Image Processing,2021,30:4828-4839.
[18]SHAO M,LI L,WANG H,et al.Selective generative adversarial network for raindrop removal from a single image[J].Neurocomputing,2021,426:265-273.
[19]YAN X,LOKE Y R.RainGAN:Unsupervised Raindrop Remo-val via Decomposition and Composition[C]//Proceedings of the IEEE Winter Conference on Applications of Computer Vision.Waikoloa,USA,2022:14-23.
[20]LIU X,SUGANUMA M,SUN Z,et al.Dual residual networks leveraging the potential of paired operations for image restoration[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Long Beach,California,USA,2019:7007-7016.
[21]GUO Y,CHEN J,REN X,et al.Joint raindrop and haze removal from a single image[J].IEEE Transactions on Image Proces-sing,2020,29:9508-9519.
[22]ZHANG K,LI D,LUO W,et al.Dual attention-in-attentionmodel for joint rain streak and raindrop removal[J].IEEE Transactions on Image Processing,2021,30:7608-7619.
[1] GE Huibin, WANG Dexin, ZHENG Tao, ZHANG Ting, XIONG Deyi. Study on Model Migration of Natural Language Processing for Domestic Deep Learning Platform [J]. Computer Science, 2024, 51(1): 50-59.
[2] JING Yeyiran, YU Zeng, SHI Yunxiao, LI Tianrui. Review of Unsupervised Domain Adaptive Person Re-identification Based on Pseudo-labels [J]. Computer Science, 2024, 51(1): 72-83.
[3] JIN Yu, CHEN Hongmei, LUO Chuan. Interest Capturing Recommendation Based on Knowledge Graph [J]. Computer Science, 2024, 51(1): 133-142.
[4] SUN Shukui, FAN Jing, SUN Zhongqing, QU Jinshuai, DAI Tingting. Survey of Image Data Augmentation Techniques Based on Deep Learning [J]. Computer Science, 2024, 51(1): 150-167.
[5] WANG Weijia, XIONG Wenzhuo, ZHU Shengjie, SONG Ce, SUN He, SONG Yulong. Method of Infrared Small Target Detection Based on Multi-depth Feature Connection [J]. Computer Science, 2024, 51(1): 175-183.
[6] SHI Dianxi, LIU Yangyang, SONG Linna, TAN Jiefu, ZHOU Chenlei, ZHANG Yi. FeaEM:Feature Enhancement-based Method for Weakly Supervised Salient Object Detection via Multiple Pseudo Labels [J]. Computer Science, 2024, 51(1): 233-242.
[7] ZHOU Wenhao, HU Hongtao, CHEN Xu, ZHAO Chunhui. Weakly Supervised Video Anomaly Detection Based on Dual Dynamic Memory Network [J]. Computer Science, 2024, 51(1): 243-251.
[8] HOU Jing, DENG Xiaomei, HAN Pengwu. Survey on Domain Limited Relation Extraction [J]. Computer Science, 2024, 51(1): 252-265.
[9] YAN Zhihao, ZHOU Zhangbing, LI Xiaocui. Survey on Generative Diffusion Model [J]. Computer Science, 2024, 51(1): 273-283.
[10] ZHAO Mingmin, YANG Qiuhui, HONG Mei, CAI Chuang. Smart Contract Fuzzing Based on Deep Learning and Information Feedback [J]. Computer Science, 2023, 50(9): 117-122.
[11] LI Haiming, ZHU Zhiheng, LIU Lei, GUO Chenkai. Multi-task Graph-embedding Deep Prediction Model for Mobile App Rating Recommendation [J]. Computer Science, 2023, 50(9): 160-167.
[12] HUANG Hanqiang, XING Yunbing, SHEN Jianfei, FAN Feiyi. Sign Language Animation Splicing Model Based on LpTransformer Network [J]. Computer Science, 2023, 50(9): 184-191.
[13] ZHU Ye, HAO Yingguang, WANG Hongyu. Deep Learning Based Salient Object Detection in Infrared Video [J]. Computer Science, 2023, 50(9): 227-234.
[14] ZHANG Yian, YANG Ying, REN Gang, WANG Gang. Study on Multimodal Online Reviews Helpfulness Prediction Based on Attention Mechanism [J]. Computer Science, 2023, 50(8): 37-44.
[15] SONG Xinyang, YAN Zhiyuan, SUN Muyi, DAI Linlin, LI Qi, SUN Zhenan. Review of Talking Face Generation [J]. Computer Science, 2023, 50(8): 68-78.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.