计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (1): 190-197.doi: 10.11896/jsjkx.230500125

• 计算机图形学&多媒体 • 上一篇    下一篇

雨滴实地拍摄基准图像数据集及评估

陈天一1, 薛文1, 全宇晖1, 许勇1,2   

  1. 1 华南理工大学计算机科学与工程学院 广州510006
    2 鹏城实验室 广东 深圳518005
  • 收稿日期:2023-05-21 修回日期:2023-09-28 出版日期:2024-01-15 发布日期:2024-01-12
  • 通讯作者: 全宇晖(csyhquan@scut.edu.cn)
  • 作者简介:(csttychen@mail.scut.edu.cn)
  • 基金资助:
    国家自然科学基金(62072188)

Raindrop In-Situ Captured Benchmark Image Dataset and Evaluation

CHEN Tianyi1, XUE Wen1, QUAN Yuhui1, XU Yong1,2   

  1. 1 School of Computer Science and Engineering,South China University of Technology,Guangzhou 510006,China
    2 Peng Cheng Laboratory,Shenzhen,Guangdong 518055,China
  • Received:2023-05-21 Revised:2023-09-28 Online:2024-01-15 Published:2024-01-12
  • About author:CHEN Tianyi,born in 1997,Ph.D candidate,is a student member of CCF(No.H5052G).His main research interests include computer vision,image proces-sing and image generation.
    QUAN Yuhui,born in 1985,Ph.D,associate professor,is a member of CCF(No.70281S).His main research in-terests include image processing,deep learning and sparse representation.
  • Supported by:
    National Natural Science Foundation of China(62072188).

PDF (PC)

1527

摘要： 在雨天透过玻璃窗拍摄时,附着在玻璃表面的雨滴通常会出现在图像中,这不仅降低了图像的可见度,还会使许多计算视觉算法无法正常工作。图像雨滴去除研究,是指从这类雨天图像中去除雨滴的具体科研研究。该研究领域面临着很大的挑战,主要原因是自然界中的雨滴形态多种多样、各不相同,不同透明度的雨滴也会影响背景图像的成像质量,从而增加了识别并去除雨滴的困难度,对去雨滴算法的性能提升造成了负面影响。为了方便研究者全面了解该领域,将从以下两个方面详尽介绍单幅图像去雨滴研究:单幅图像去雨滴算法和单幅图像联合去雨算法;同时也对该领域的所有算法进行了总结与评估。在基于深度学习的方法中,算法的性能往往受限于数据集的质量,但现有的雨滴数据集中均存在雨滴图像质量不高、图像数量不足等常见情况。为此,建立了雨滴实地拍摄基准图像数据集(HEMC),在拍摄过程中,尽量避免相机抖动、窗户反射和其他外界条件的干扰,从而提高了数据集中训练集的图像质量和测试集的精准度,进而间接提升了算法性能。同时,利用主观视觉效果以及客观指标对数据集进行了多方面的评估,实验结果展现了HEMC数据集中图像的多样性以及客观指标的稳定性。此外,通过对雨滴数据集间的交叉验证,证实了HEMC数据集在已有去雨滴算法中的通用性与稳定性。

关键词: 图像去雨滴, 雨滴图像数据集, 深度学习, 图像评价指标

Abstract: When taking photos through glass windows in rainy days,the raindrops adhered to glass surfaces are usually presented in the images,which not only degrade the visibility of the image but also prevent many computer vision algorithms from functioning properly.The research on raindrop removal is a scientific research to remove raindrops from such rainy images.The singleimage raindrop removal research presents significant challenges due to the diverse and unique forms of raindrops found in nature.The varying transparency of raindrops further complicates the task of removing raindrop artifacts and degrades the imaging quality of background scenes,adversely impacting the performance of existing raindrop removal algorithms.To facilitate a comprehensive understanding of this research area,this paper provides a detailed introduction to single-image raindrop removal,covering two main aspects:single-image raindrop removal algorithms and joint raindrop removal algorithms for single images.Additionally,a summary and evaluation of existing algorithms in this field are presented.However,the performance of the algorithm is often li-mited by the quality and quantity of the dataset in deep learning based methods,but in existing raindrop datasets,common situations such as low-quality raindrop images and insufficient image quantities exist.In existing raindrop datasets,there are common situations such as poor quality of raindrop images and insufficient number of raindrop images.This paper proposes a higher education megacenter(HEMC) dataset.Camera shake,window reflections and other external disturbances are avoided as much as possible thus improving the image quality of the training set and accuracy of the test set and indirectly improving the performance of the raindrop removal methods.HEMC is evaluated in various aspects using competent visual effects and objective metrics.Experimental results show the diversity of the raindrop images in HEMC and stability of the objective metrics.In addition,the results verify the universality and stability of the HEMC in the raindrop removal methods.

Key words: Image raindrop removal, Raindrop image dataset, Deep learning, Image evaluation metric

中图分类号: 

  • TP391.4
[1]YOU S,TAN R,KAWAKAMI R,et al.Adherent raindrop detection and removal in video[C]//Proceedings of the IEEE Conference Computer Vision Pattern Recognition.Portland,USA,2013:1035-1042.
[2]QIAN R,TAN R,YANG W,et al.Attentive generative adversarial network for raindrop removal from a single image[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Salt Lake City,USA,2018:2482-2491.
[3]QUAN Y,DENG S,CHEN Y,et al.Deep learning for seeingthrough window with raindrops[C]//Proceedings of the IEEE International Conference Computer Vision.Seoul,Korea(South),2019:2463-2471.
[4]QUAN R,YU X,LIANG Y,et al.Removing raindrops and rain streaks in one go[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Virtual.2021:9147-9156.
[5]MIAO Y,HONG H,KIM H.Size and angle filter based rain removal in video for outdoor surveillance systems[C]//IEEE Asian Control Conference.Taipei,2011:1300-1304.
[6]EIGEN D,KRISHNAN D,FERGUS R,et al.Restoring animage taken through a window covered with dirt or rain[C]//Proceedings of the IEEE International Conference on Computer Vision.Sydney,Australia,2013:633-640.
[7]ISOLA P,ZHU J Y,ZHOU T,et al.Image-to-image translation with conditional adversarial networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Honolulu,Hawaii,2017:1125-1134.
[8]PENG J,XU Y,CHEN T,et al.Single-image raindrop removal using concurrent channel-spatial attention and long-short skip connections[J].Pattern Recognition Letters,2020,131:121-127.
[9]HE D,SHANG X,LUO J,et al.Adherent mist and raindrop removal from a single image using attentive convolutional network[J].Neurocomputing,2022,505:178-187.
[10]WEI Y,ZHANG Z,XU M,et al.Robust attention deraining network for synchronous rain streaks and raindrops removal[C]//Proceedings of the 30th ACM International Conference on Multimedia.Lisbon,Portugal,2022:6464-6472.
[11]RONNEBERGER O,FISCHER P,BROX T,et al.U-net:Con-volutional networks for biomedical image segmentation[C]//Proceedings of the International Conference on MedicalImage Computing and Computer-assisted Intervention.Munich,Germany,2015:234-241.
[12]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas,USA,2016:770-778.
[13]HAO Z,YOU S,LI Y,et al.Learning from synthetic photorea-listic raindrop for single image raindrop removal[C]//Procee-dings of the IEEE International Conference on Computer Vision Workshops.Seoul,Korea(South),2019.
[14]ALLETTO S,CARLIN C,RIGAZIO L,et al.Adherent raindrop removal with self-supervised attention maps and spatio-temporal generative adversarial networks[C]//Proceedings of the IEEEInternational Conference on Computer Vision Workshops.Seoul,Korea(South),2019.
[15]REN D,LI J,HAN M,et al.Not All Areas Are Equal:A Novel Separation-Restoration-Fusion Network for Image Raindrop Removal[J].Computer Graphics Forum,2020,39(7):495-505.
[16]LUO W,LAI J,XIE X,et al.Weakly supervised learning for raindrop removal on a single image[J].IEEE Transactions on Circuits and Systems for Video Technology,2020,31(5):1673-1683.
[17]SHAO M W,LI L,MENG D Y,et al.Uncertainty guided multi-scale attention network for raindrop removal from a single image[J].IEEE Transactions on Image Processing,2021,30:4828-4839.
[18]SHAO M,LI L,WANG H,et al.Selective generative adversarial network for raindrop removal from a single image[J].Neurocomputing,2021,426:265-273.
[19]YAN X,LOKE Y R.RainGAN:Unsupervised Raindrop Remo-val via Decomposition and Composition[C]//Proceedings of the IEEE Winter Conference on Applications of Computer Vision.Waikoloa,USA,2022:14-23.
[20]LIU X,SUGANUMA M,SUN Z,et al.Dual residual networks leveraging the potential of paired operations for image restoration[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Long Beach,California,USA,2019:7007-7016.
[21]GUO Y,CHEN J,REN X,et al.Joint raindrop and haze removal from a single image[J].IEEE Transactions on Image Proces-sing,2020,29:9508-9519.
[22]ZHANG K,LI D,LUO W,et al.Dual attention-in-attentionmodel for joint rain streak and raindrop removal[J].IEEE Transactions on Image Processing,2021,30:7608-7619.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发