计算机科学 ›› 2025, Vol. 52 ›› Issue (11): 113-122.doi: 10.11896/jsjkx.241200176

• 计算机图形学&多媒体 • 上一篇    下一篇

FE-DARFormer:基于频域增强与退化感知路由Transformer的图像去雪模型

秦溢, 战鹏祥, 鲜峰, 柳晨龙, 王明辉   

  1. 四川大学计算机学院 成都 610065
  • 收稿日期:2024-12-24 修回日期:2025-03-09 出版日期:2025-11-15 发布日期:2025-11-06
  • 通讯作者: 王明辉(wangminghui@scu.edu.cn)
  • 作者简介:(2022223045231@stu.scu.edu.cn)
  • 基金资助:
    国家重点研发计划(2022YFC2407604)

FE-DARFormer:Image Desnowing Model Based on Frequency Enhancement and Degradation- aware Routing Transformer

QIN Yi, ZHAN Pengxiang, XIAN Feng, LIU Chenlong, WANG Minghui   

  1. College of Computer Science,Sichuan University,Chengdu 610065,China
  • Received:2024-12-24 Revised:2025-03-09 Online:2025-11-15 Published:2025-11-06
  • About author:QIN Yi,born in 1996,master.His main research interests include image processing and computer vision.
    WANG Minghui,born in 1971,Ph.D,professor.His main research interests include intelligent medical image processing and computer vision.
  • Supported by:
    National Key Research and Development Program of China(2022YFC2407604).

摘要: 图像去雪的目标是从包含复杂雪景退化的图像中恢复清晰的场景信息。与雨的规律性和半透明性不同,雪具有各种退化形态和尺度,严重退化的区域会严重遮挡场景信息。近年来,许多方法通过自注意力机制来恢复不同的退化现象。然而,对图像所有区域进行全局自注意力计算成本较高。为了降低计算成本,这些方法通常将自注意力计算限制在有限的窗口内。但是由于严重退化区域的遮挡效应,这些退化区域的恢复只能依赖捕捉周围区域的信息,对图像进行恢复时严重退化区域受到感受野瓶颈的限制,难以聚合更多信息。因此,这些方法难以有效恢复大面积退化的区域。为了进一步提升去雪性能,提出了一种新颖有效的去雪方法。从退化感知路由与频域增强的角度出发,提出了退化感知路由Transformer和双频域增强Transformer,并将两者结合,提出了新的网络架构——FE-DARFormer。FE-DARFormer能够针对严重退化区域进行动态路由和全局自注意力计算,从而获得全局感受野,有效恢复大面积退化区域,并降低计算成本。此外,该方法能通过离散小波分解出高低频信息,从而有效恢复多尺度的雪景退化并识别多样化的雪花形态与纹理特征。

关键词: 图像去雪, 退化感知路由, 双频域增强, 全局感受野

Abstract: The goal of image desnowing is to restore clear scene information from images degraded by complex snowy scenes.Unlike the regularity and semi-transparency of rain,snow exhibits various forms and scales of degradation,with severely degraded regions often obstructing important scene details.Recent methods have employed self-attention mechanisms to address different degradation phenomena.However,global self-attention computation across all image regions is computationally expensive,leading these methods to restrict attention to smaller windows.Yet,due to the occlusion effects in severely degraded areas,the recovery of these regions relies heavily on capturing information from surrounding areas,which results in a receptive field bottleneck,limi-ting the ability to aggregate sufficient information.As a result,these methods struggle to effectively restore large-scale degraded regions.To improve desnowing performance,this paper proposes a novel approach,introducing a new network architecture called FE-DARFormer,which combines a Degradation-Aware Routing Transformer and a Dual-Frequency Enhancement Transformer.FE-DARFormer dynamically routes and applies global self-attention to severely degraded regions,enabling a global receptive field for effective restoration of large degraded areas while reducing computational cost.Additionally,it uses discrete wavelet decomposition to handle multi-scale snow degradation,enhancing the recovery of diverse snowflake shapes and textures.

Key words: Image desnowing, Degradation-aware routing, Dual-frequency enhancement, Global receptive field

中图分类号: 

  • TP391
[1]YANG R,LI W,SHANG X,et al.KPE-YOLOv5:an improved small target detection algorithm based on YOLOv5[J].Electronics,2023,12(4):817.
[2]FAN J,YANG X,LU R,et al.Long-term visual tracking algorithm for UAVs based on kernel correlation filtering and SURF features[J].The Visual Computer,2023,39(1):319-333.
[3]CHANG Y,TU Z,XIE W,et al.Video anomaly detection with spatio-temporal dissociation[J].Pattern Recognition,2022,122:108213.
[4]LIU Y,YANG D,WANG Y,et al.Generalized video anomalyevent detection:Systematic taxonomy and comparison of deep models[J].ACM Computing Surveys,2024,56(7):1-38.
[5]CHEN W T,FANG H Y,DING J J,et al.JSTASR:Joint size and transparency-aware snow removal algorithm based on modified partial convolution and veiling effect removal[C]//Compu-ter Vision-ECCV 2020:16th European Conference.2020:754-770.
[6]PEI S C,TSAI Y T,LEE C Y.Removing rain and snow in a single image using saturation and visibility features[C]//2014 IEEE International Conference on Multimedia and Expo Workshops(ICMEW).IEEE,2014:1-6.
[7]ZHENG X,LIAO Y,GUO W,et al.Single-image-based rain and snow removal using multi-guided filter[C]//Neural Information Processing:20th International Conference,ICONIP 2013.Berlin:Springer,2013:258-265.
[8]VORONIN V,SEMENISHCHEV E,ZHDANOVA M,et al.Rain and snow removal using multi-guided filter and anisotropic gradient in the quaternion framework[C]//Artificial Intelligence and Machine Learning in Defense Applications.SPIE,2019:227-233.
[9]HUANG J,LIU Y,FU X,et al.Exposure normalization andcompensation for multiple-exposure correction[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:6043-6052.
[10]HUANG J,ZHAO F,ZHOU M,et al.Learning sample relationship for exposure correction[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023:9904-9913.
[11]VALANARASU J M J,YASARLA R,PATEL V M.Trans-weather:Transformer-based restoration of images degraded by adverse weather conditions[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:2353-2363.
[12]LIU Y F,JAW D W,HUANG S C,et al.Desnownet:Context-aware deep network for snow removal[J].IEEE Transactions on Image Processing,2018,27(6):3064-3073.
[13]LI B,REN W,FU D,et al.Benchmarking single-image dehazing and beyond[J].IEEE Transactions on Image Processing,2018,28(1):492-505.
[14]CHEN W T,FANG H Y,HSIEH C L,et al.All snow removed:Single image desnowing algorithm using hierarchical dual-tree complex wavelet representation and contradict channel loss[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:4196-4205.
[15]ZAMIR S W,ARORA A,KHAN S,et al.Restormer:Efficient transformer for high-resolution image restoration[C]//Procee-dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:5728-5739.
[16]ALEXEY D.An image is worth 16x16 words:Transformers for image recognition at scale[J].arXiv:2010.11929,2020.
[17]LIU Z,LIN Y,CAO Y,et al.Swin transformer:Hierarchical vision transformer using shifted windows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:10012-10022.
[18]SONG Y,HE Z,QIAN H,et al.Vision transformers for single image dehazing[J].IEEE Transactions on Image Processing,2023,32:1927-1941.
[19]CUI Y,REN W,CAO X,et al.Focal network for image restoration[C]//Proceedings of the IEEE/CVF International Confe-rence on Computer Vision.2023:13001-13011.
[20]CHEN S,YE T,LIU Y,et al.Dual-former:Hybrid self-attention transformer for efficient image restoration[J].Digital Signal Processing,2024,149:104485.
[21]MALLAT S.A wavelet tour of signal processing[M].Academic Press,1999.
[22]YU Z,ZHAO C,WANG Z,et al.Searching central differenceconvolutional networks for face anti-spoofing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:5295-5305.
[23]BOSSU J,HAUTIERE N,TAREL J P.Rain or snow detection in image sequences through use of a histogram of orientation of streaks[J].International Journal of Computer Vision,2011,93:348-367.
[24]HUANG L W,CUI W C,SHAO H.Research on Pedestrian Detection Method Based on Multi-Layer Feature Fusion[J].Computer Science,2024,51(S2):489-495.
[25]LIU Z K,YIN J B.Text-Driven Emotional Diversification in Facial Animation Generation[J].Computer Science,2024,51(S2):323-330.
[26]LI W X,ZHANG J,ZHUO L,et al.Advances in Vision Segmentation Techniques Based on Transformer[J].Journal of Computer Science and Technology,2024,47(12):2760-2782.
[27]LI T,ZHAO E D,YANG J.Road Obstacle Detection MethodBased on Self-Attention and Bidirectional Feature Fusion[J].Computer Science,2024,51(S2):287-291.
[28]JIANG C J,HE X Y,XIANG J.LOL-YOLO:Low-Light Object Detection with Multi-Attention Mechanism Fusion[J].Compu-ter Engineering and Applications,2024,60(24):177-187.
[29]TANG S,ZENG W L,YANG S L,et al.Single Image Super-Resolution Reconstruction Network Based on Transformer with Block-Intra and Block-Inter Dual Aggregation[J].Journal of Computer Science and Technology,2024,47(12):2783-2802.
[30]YANG Y D,GE H B,XIN S A,et al.Lightweight Remote Sen-sing Image Small Object Detection Fusing Super-Resolution and Feature Enhancement[J].Computer Engineering,2024,50(11):284-296.
[31]REN J,ZHOU G,ZHU Y,et al.Unsupervised Domain Adaptive Learning for Image Desnowing with Real-World Data[C]//2023 IEEE International Conference on Image Processing(ICIP).IEEE,2023:3050-3054.
[32]YASARLA R,SINDAGI V A,PATEL V M.Unsupervised restoration of weather-affected images using deep gaussian process-based cyclegan[C]//2022 26th International Conference on Pattern Recognition(ICPR).IEEE,2022:1967-1974.
[33]ZHANG K,LI R,YU Y,et al.Deep dense multi-scale network for snow removal using semantic and depth priors[J].IEEE Transactions on Image Processing,2021,30:7419-7431.
[34]CUI Y,REN W,CAO X,et al.Revitalizing convolutional net-work for image restoration[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2024,46(12): 9423-9438.
[35]LIANG J,CAO J,SUN G,et al.Swinir:Image restoration using swin transformer[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:1833-1844.
[36]WANG Z,CUN X,BAO J,et al.Uformer:A general u-shaped transformer for image restoration[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:17683-17693.
[37]CHEN S,YE T,LIU Y,et al.CPLFormer:Cross-scale Proto-type Learning Transformer for Image Snow Removal[C]//Proceedings of the 31st ACM International Conference on Multimedia.2023:4228-4239.
[38]XU F,NARESH BODDETI V,SAVVIDES M.Local binaryconvolutional neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:19-28.
[39]RONNEBERGER O,FISCHER P,BROX T.U-Net:Convolu-tional networks for biomedical image segmentation[C]//Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015.2015:234-241.
[40]WANG X,GIRSHICK R,GUPTA A,et al.Non-local neuralnetworks[C]//Proceedings of the IEEE Conference on Compu-ter Vision and Pattern Recognition.2018:7794-7803.
[41]JIANG L,DAI B,WU W,et al.Focal frequency loss for image reconstruction and synthesis[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:13919-13929.
[42]KINGMA D P.Adam:A method for stochastic optimization[J].arXiv:1412.6980,2014.
[43]CHEN L,CHU X,ZHANG X,et al.Simple baselines for image restoration[C]//European Conference on Computer Vision.Cham:Springer,2022:17-33.
[44]MOU C,WANG Q,ZHANG J.Deep generalized unfolding networks for image restoration[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:17399-17410.
[45]CHEN W T,HUANG Z K,TSAI C C,et al.Learning multiple adverse weather removal via two-stage knowledge learning and multi-contrastive regularization:Toward a unified model[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:17653-17662.
[46]CHENG B,LI J,CHEN Y,et al.Snow mask guided adaptive residual network for image snow removal[J].Computer Vision and Image Understanding,2023,236:103819.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!