计算机科学 ›› 2025, Vol. 52 ›› Issue (11): 206-212.doi: 10.11896/jsjkx.240900013

• 计算机图形学&多媒体 • 上一篇    下一篇

基于联合注意力机制与多阶段特征提取的图像去雨

林祖凯, 侯国家, 王国栋, 潘振宽   

  1. 青岛大学计算机科学技术学院 山东 青岛 266071
  • 收稿日期:2024-09-02 修回日期:2024-11-26 出版日期:2025-11-15 发布日期:2025-11-06
  • 通讯作者: 侯国家(guojiahou@qdu.edu.cn)
  • 作者简介:(linzukai@qdu.edu.cn)
  • 基金资助:
    山东省自然科学基金(ZR2024MF125);青岛市自然科学基金(24-4-4-zrjj-122-jch);国家自然科学基金(61901240)

Image Deraining Based on Union Attention Mechanism and Multi-stage Feature Extraction

LIN Zukai, HOU Guojia, WANG Guodong, PAN Zhenkuan   

  1. College of Computer Science & Technology,Qingdao University,Qingdao,Shandong 266071,China
  • Received:2024-09-02 Revised:2024-11-26 Online:2025-11-15 Published:2025-11-06
  • About author:LIN Zukai,born in 2000,postgraduate,is a member of CCF(No.U6520G).His main research interest is image proces-sing.
    HOU Guojia,born in 1986,Ph.D,asso-ciate professor,master supervisor,is a member of CCF(No.76713M).His main research interests include image processing and pattern recognition.
  • Supported by:
    Natural Science Foundation of Shandong Province(ZR2024MF125),Natural Science Foundation of Qingdao Municipality(24-4-4-zrjj-122-jch) and National Natural Science Foundation of China(61901240).

摘要: 现有的图像去雨网络主要依赖大量合成配对数据进行训练,忽视了合成数据与真实数据在空间分布特征和通道重要性上的差异,导致在真实数据上的去雨效果存在纹理细节模糊和泛化性差等问题。为此,提出了一种基于联合注意力机制与多阶段特征提取的无监督图像去雨网络。首先,为了适应雨纹的空间位置局部性,设计了结合空间和通道注意力机制的雨纹特征感知模块,并通过扩张卷积增大雨纹特征提取感受野。其次,引入循环神经网络渐进地分阶段提取雨纹特征,并在循环中保留前一阶段的有用信息,以增强对雨纹特征的提取能力。为了进一步提升对局部微观细节和全局纹理结构特征的鉴别能力,设计了一个多尺度鉴别器,分别在3个不同尺度上对生成图像进行判别,以指导生成器生成更高质量的图像。在合成和真实数据集上进行了定性和定量实验,通过PSNR,SSIM和NIQE客观评价指标对比表明,所提出方法的结果优于对比的监督、半监督和无监督方法,验证了其有效性和泛化性。

关键词: 图像处理, 图像去雨, 联合注意力机制, 多阶段特征提取, 无监督学习

Abstract: Existing image deraining networks predominantly rely on the large-scale synthetic paired datasets for training,ignoring the difference in spatial distribution characteristics and the difference in channel importance between synthetic and real data,resulting in blurred texture details and diminished generalization performance.To address these issues,this paper develops an unsupervised network model for image deraining based on a union attention mechanism with multi-stage feature extraction.To adapt to the spatial locality of rain streaks,the feature-aware module is initially designed to extract rain streaks through the combination of spatial and channel attention mechanisms,while dilation convolution is used to enhance the sensory field of rain feature extraction.In addition,a recurrent neural network is introduced to extract the rain stripe features gradually,and the useful information of the previous stage is retained in the cycle to improve the rain stripe feature extraction ability.To further enhance the discrimination of local micro-details and global texture structure features,it designs a multi-scale discriminator for distinguishing images at three different scales and guideings the generator to produce higher quality images.Qualitative and quantitative experiments on synthetic and real datasets show that the proposed method is superior to some supervised,semi-supervised and unsupervised me-thods on PSNR,SSIM and NIQE metrics,which verifies its effectiveness and generalization.

Key words: Image processing, Image deraining, Multi-attention mechanism, Multi-stage feature extraction, Unsupervised learning

中图分类号: 

  • TP391
[1]ZHAO Z Q,ZHENG P,XU S T,et al.Object detection withdeep learning:A review[J].IEEE Transactions on Neural Networks and Learning Systems,2019,30(11):3212-3232.
[2]YU J,TAN M,ZHANG H Y,et al.Hierarchical deep click feature prediction for fine-grained image recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2022,44(2):563-578.
[3]MARVASTI-ZADEH S M,CHENG L,GHANEI-YAKHDANH,et al.Deep learning for visual tracking:A comprehensive survey[J].IEEE Transactions on Intelligent Transportation Systems,2022,23(5):3943-3968.
[4]YANG W H,TAN R T,WANG S Q,et al.Single image derai-ning:From model-based to data-driven and beyond[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2021,43(11):4059-4077.
[5]ZHANG Z,WEI Y Y,ZHANG H J,et al.Data-driven single image deraining:A comprehensive review and new perspectives[J].Pattern Recognition,2023,143:109740.
[6]MENG X Y,XUE X H,LI W L,et al.Motion-estimation based space-temporal feature aggregation network for multi-frames rain removal[J].Computer Science,2021,48(5):170-176.
[7]XIE Q Q,ZHANG H,GAI S.Multi-resolution context aggregation network for single image rain removal[J].Journal of Computer-Aided Design & Conputer Graphics,2022,34(2):232-244.
[8]WEI Y Y,ZHANG Z,WANG Y,et al.Deraincyclegan:Rain at-tentive cyclegan for single image deraining and rainmaking[J].IEEE Transactions on Image Processing,2021,30:4788-4801.
[9]CHEN M M,WANG P,SHANG D H,et al.Cycle-attention-derain:Unsupervised rain removal with CycleGAN[J].The Visual Computer,2023,39(8):3727-3739.
[10]KANG L W,LIN C W,FU Y H.Automatic single-image-based rain streaks removal via image decomposition[J].IEEE Transactions on Image Processing,2012,21(4):1742-1755.
[11]DU S L,LIU Y G,YE M,et al.Single image deraining via decorrelating the rain streaks and background scene in gradient domain[J].Pattern Recognition,2018,79:303-317.
[12]WANG H,XIE Q,ZHAO Q,et al.A Model-Driven Deep Neural Network for Single Image Rain Removal[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:3103-3112.
[13]HSU W Y,CHANG W C.Recurrent wavelet structure-preser-ving residual network for single image deraining[J].Pattern Recognition,2023,137:109294.
[14]CHEN X,PAN J H,DONG J.Bidirectional Multi-Scale Implicit Neural Representations for Image Deraining[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2024:25627-25636.
[15]QIU Y R,YAO G L,FENG J,et al.Single image de-raining algorithm based on semi-supervised learning[J].Journal of Computer Applications,2022,42(5):1577-1582.
[16]JIANG N F,LUO J W,LIN J H,et al.Lightweight semi-supervised network for single image rain removal[J].Pattern Recognition,2023,137:109277.
[17]CHEN X,PAN J S,JIANG K,et al.Unpaired Deep Image Deraining Using Dual Contrastive Learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022:2017-2026.
[18]YASARLA R,SINDAGI V A,PATEL V M.Syn2real Transfer Learning for Image Deraining Using Gaussian Processes[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:2726-2736.
[19]YE Y T,YU C F,CHANG Y,et al.Unsupervised Deraining:Where Contrastive Learning Meets Self-Similarity[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:5821-5830.
[20]WEN Y B,GAO T,CHEN T.Neural schrödinger bridge for unpaired real-world image deraining[J].Information Sciences,2024,682:121199.
[21]YANG W H,TAN R T,FENG J S,et al.Deep Joint Rain Detection and Removal from a Single Image[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:1357-1366.
[22]HU J,SHEN L,SUN G.Squeeze-and-Excitation Networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:7132-7141.
[23]JADERBERG M,SIMONYAN K,ZISSERMAN A,et al.Spatial transformer networks[C]//Proceedings of the 29th InternationalConference on Neural Information Processing Systems.2015:2017-2025.
[24]LI Y,TAN R T,GUO X J,et al.Rain Streak Removal Using Layer Priors[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:2736-2744.
[25]WANG T Y,YANG X,XU K,et al.Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:12270-12279.
[26]GUO Y,XIAO X Y,ChANG Y,et al.From Sky to the Ground:A Large-Scale Benchmark and Simple Baseline Towards Real Rain Removal[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2023:12097-12107.
[27]ZHANG H,JIA D L,HAN Z X.Dual-task complementary networks for single-image deraining[J].Signal,Image and Video Processing,2023,17(8):4171-4179.
[28]ZHU J Y,PARK T,ISOLA P,et al.Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:2223-2232.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!