Computer Science ›› 2025, Vol. 52 ›› Issue (11): 113-122.doi: 10.11896/jsjkx.241200176

• Computer Graphics & Multimedia • Previous Articles     Next Articles

FE-DARFormer:Image Desnowing Model Based on Frequency Enhancement and Degradation- aware Routing Transformer

QIN Yi, ZHAN Pengxiang, XIAN Feng, LIU Chenlong, WANG Minghui   

  1. College of Computer Science,Sichuan University,Chengdu 610065,China
  • Received:2024-12-24 Revised:2025-03-09 Online:2025-11-15 Published:2025-11-06
  • About author:QIN Yi,born in 1996,master.His main research interests include image processing and computer vision.
    WANG Minghui,born in 1971,Ph.D,professor.His main research interests include intelligent medical image processing and computer vision.
  • Supported by:
    National Key Research and Development Program of China(2022YFC2407604).

Abstract: The goal of image desnowing is to restore clear scene information from images degraded by complex snowy scenes.Unlike the regularity and semi-transparency of rain,snow exhibits various forms and scales of degradation,with severely degraded regions often obstructing important scene details.Recent methods have employed self-attention mechanisms to address different degradation phenomena.However,global self-attention computation across all image regions is computationally expensive,leading these methods to restrict attention to smaller windows.Yet,due to the occlusion effects in severely degraded areas,the recovery of these regions relies heavily on capturing information from surrounding areas,which results in a receptive field bottleneck,limi-ting the ability to aggregate sufficient information.As a result,these methods struggle to effectively restore large-scale degraded regions.To improve desnowing performance,this paper proposes a novel approach,introducing a new network architecture called FE-DARFormer,which combines a Degradation-Aware Routing Transformer and a Dual-Frequency Enhancement Transformer.FE-DARFormer dynamically routes and applies global self-attention to severely degraded regions,enabling a global receptive field for effective restoration of large degraded areas while reducing computational cost.Additionally,it uses discrete wavelet decomposition to handle multi-scale snow degradation,enhancing the recovery of diverse snowflake shapes and textures.

Key words: Image desnowing, Degradation-aware routing, Dual-frequency enhancement, Global receptive field

CLC Number: 

  • TP391
[1]YANG R,LI W,SHANG X,et al.KPE-YOLOv5:an improved small target detection algorithm based on YOLOv5[J].Electronics,2023,12(4):817.
[2]FAN J,YANG X,LU R,et al.Long-term visual tracking algorithm for UAVs based on kernel correlation filtering and SURF features[J].The Visual Computer,2023,39(1):319-333.
[3]CHANG Y,TU Z,XIE W,et al.Video anomaly detection with spatio-temporal dissociation[J].Pattern Recognition,2022,122:108213.
[4]LIU Y,YANG D,WANG Y,et al.Generalized video anomalyevent detection:Systematic taxonomy and comparison of deep models[J].ACM Computing Surveys,2024,56(7):1-38.
[5]CHEN W T,FANG H Y,DING J J,et al.JSTASR:Joint size and transparency-aware snow removal algorithm based on modified partial convolution and veiling effect removal[C]//Compu-ter Vision-ECCV 2020:16th European Conference.2020:754-770.
[6]PEI S C,TSAI Y T,LEE C Y.Removing rain and snow in a single image using saturation and visibility features[C]//2014 IEEE International Conference on Multimedia and Expo Workshops(ICMEW).IEEE,2014:1-6.
[7]ZHENG X,LIAO Y,GUO W,et al.Single-image-based rain and snow removal using multi-guided filter[C]//Neural Information Processing:20th International Conference,ICONIP 2013.Berlin:Springer,2013:258-265.
[8]VORONIN V,SEMENISHCHEV E,ZHDANOVA M,et al.Rain and snow removal using multi-guided filter and anisotropic gradient in the quaternion framework[C]//Artificial Intelligence and Machine Learning in Defense Applications.SPIE,2019:227-233.
[9]HUANG J,LIU Y,FU X,et al.Exposure normalization andcompensation for multiple-exposure correction[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:6043-6052.
[10]HUANG J,ZHAO F,ZHOU M,et al.Learning sample relationship for exposure correction[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023:9904-9913.
[11]VALANARASU J M J,YASARLA R,PATEL V M.Trans-weather:Transformer-based restoration of images degraded by adverse weather conditions[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:2353-2363.
[12]LIU Y F,JAW D W,HUANG S C,et al.Desnownet:Context-aware deep network for snow removal[J].IEEE Transactions on Image Processing,2018,27(6):3064-3073.
[13]LI B,REN W,FU D,et al.Benchmarking single-image dehazing and beyond[J].IEEE Transactions on Image Processing,2018,28(1):492-505.
[14]CHEN W T,FANG H Y,HSIEH C L,et al.All snow removed:Single image desnowing algorithm using hierarchical dual-tree complex wavelet representation and contradict channel loss[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:4196-4205.
[15]ZAMIR S W,ARORA A,KHAN S,et al.Restormer:Efficient transformer for high-resolution image restoration[C]//Procee-dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:5728-5739.
[16]ALEXEY D.An image is worth 16x16 words:Transformers for image recognition at scale[J].arXiv:2010.11929,2020.
[17]LIU Z,LIN Y,CAO Y,et al.Swin transformer:Hierarchical vision transformer using shifted windows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:10012-10022.
[18]SONG Y,HE Z,QIAN H,et al.Vision transformers for single image dehazing[J].IEEE Transactions on Image Processing,2023,32:1927-1941.
[19]CUI Y,REN W,CAO X,et al.Focal network for image restoration[C]//Proceedings of the IEEE/CVF International Confe-rence on Computer Vision.2023:13001-13011.
[20]CHEN S,YE T,LIU Y,et al.Dual-former:Hybrid self-attention transformer for efficient image restoration[J].Digital Signal Processing,2024,149:104485.
[21]MALLAT S.A wavelet tour of signal processing[M].Academic Press,1999.
[22]YU Z,ZHAO C,WANG Z,et al.Searching central differenceconvolutional networks for face anti-spoofing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:5295-5305.
[23]BOSSU J,HAUTIERE N,TAREL J P.Rain or snow detection in image sequences through use of a histogram of orientation of streaks[J].International Journal of Computer Vision,2011,93:348-367.
[24]HUANG L W,CUI W C,SHAO H.Research on Pedestrian Detection Method Based on Multi-Layer Feature Fusion[J].Computer Science,2024,51(S2):489-495.
[25]LIU Z K,YIN J B.Text-Driven Emotional Diversification in Facial Animation Generation[J].Computer Science,2024,51(S2):323-330.
[26]LI W X,ZHANG J,ZHUO L,et al.Advances in Vision Segmentation Techniques Based on Transformer[J].Journal of Computer Science and Technology,2024,47(12):2760-2782.
[27]LI T,ZHAO E D,YANG J.Road Obstacle Detection MethodBased on Self-Attention and Bidirectional Feature Fusion[J].Computer Science,2024,51(S2):287-291.
[28]JIANG C J,HE X Y,XIANG J.LOL-YOLO:Low-Light Object Detection with Multi-Attention Mechanism Fusion[J].Compu-ter Engineering and Applications,2024,60(24):177-187.
[29]TANG S,ZENG W L,YANG S L,et al.Single Image Super-Resolution Reconstruction Network Based on Transformer with Block-Intra and Block-Inter Dual Aggregation[J].Journal of Computer Science and Technology,2024,47(12):2783-2802.
[30]YANG Y D,GE H B,XIN S A,et al.Lightweight Remote Sen-sing Image Small Object Detection Fusing Super-Resolution and Feature Enhancement[J].Computer Engineering,2024,50(11):284-296.
[31]REN J,ZHOU G,ZHU Y,et al.Unsupervised Domain Adaptive Learning for Image Desnowing with Real-World Data[C]//2023 IEEE International Conference on Image Processing(ICIP).IEEE,2023:3050-3054.
[32]YASARLA R,SINDAGI V A,PATEL V M.Unsupervised restoration of weather-affected images using deep gaussian process-based cyclegan[C]//2022 26th International Conference on Pattern Recognition(ICPR).IEEE,2022:1967-1974.
[33]ZHANG K,LI R,YU Y,et al.Deep dense multi-scale network for snow removal using semantic and depth priors[J].IEEE Transactions on Image Processing,2021,30:7419-7431.
[34]CUI Y,REN W,CAO X,et al.Revitalizing convolutional net-work for image restoration[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2024,46(12): 9423-9438.
[35]LIANG J,CAO J,SUN G,et al.Swinir:Image restoration using swin transformer[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:1833-1844.
[36]WANG Z,CUN X,BAO J,et al.Uformer:A general u-shaped transformer for image restoration[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:17683-17693.
[37]CHEN S,YE T,LIU Y,et al.CPLFormer:Cross-scale Proto-type Learning Transformer for Image Snow Removal[C]//Proceedings of the 31st ACM International Conference on Multimedia.2023:4228-4239.
[38]XU F,NARESH BODDETI V,SAVVIDES M.Local binaryconvolutional neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:19-28.
[39]RONNEBERGER O,FISCHER P,BROX T.U-Net:Convolu-tional networks for biomedical image segmentation[C]//Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015.2015:234-241.
[40]WANG X,GIRSHICK R,GUPTA A,et al.Non-local neuralnetworks[C]//Proceedings of the IEEE Conference on Compu-ter Vision and Pattern Recognition.2018:7794-7803.
[41]JIANG L,DAI B,WU W,et al.Focal frequency loss for image reconstruction and synthesis[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:13919-13929.
[42]KINGMA D P.Adam:A method for stochastic optimization[J].arXiv:1412.6980,2014.
[43]CHEN L,CHU X,ZHANG X,et al.Simple baselines for image restoration[C]//European Conference on Computer Vision.Cham:Springer,2022:17-33.
[44]MOU C,WANG Q,ZHANG J.Deep generalized unfolding networks for image restoration[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:17399-17410.
[45]CHEN W T,HUANG Z K,TSAI C C,et al.Learning multiple adverse weather removal via two-stage knowledge learning and multi-contrastive regularization:Toward a unified model[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:17653-17662.
[46]CHENG B,LI J,CHEN Y,et al.Snow mask guided adaptive residual network for image snow removal[J].Computer Vision and Image Understanding,2023,236:103819.
[1] FANG Quan, ZHANG Jinlong, WANG Bingqian, HU Jun. Research on Domain Knowledge Question Answering via Large Language Models withCompositional Context Prompting [J]. Computer Science, 2025, 52(11): 13-21.
[2] PI Qiankun, LU Jicang, ZHU Taojie, PENG Yueling. Zero-shot Knowledge Extraction Method Based on Large Language Model Enhanced [J]. Computer Science, 2025, 52(11): 22-29.
[3] ZHANG Haoran, HAO Wenning, JIN Dawei, CHENG Kai, ZHAI Ying. DF-RAG:A Retrieval-augmented Generation Method Based on Query Rewriting and Knowledge Selection [J]. Computer Science, 2025, 52(11): 30-39.
[4] HU Jintao, XIAN Guangming. Self-attention-based Graph Contrastive Learning for Recommendation [J]. Computer Science, 2025, 52(11): 82-89.
[5] WANG Wenpeng, GE Hongwei, LI Ting. Adversarial Generative Multi-sensitive Attribute Data Biasing Method [J]. Computer Science, 2025, 52(11): 90-97.
[6] CHEN Kang, LIN Jianhan, LIU Yuanjie. Survey on Image Deblurring Algorithms [J]. Computer Science, 2025, 52(11): 98-112.
[7] YAN Zhilin, NIE Rencan. Infrared and Visible Image Fusion Cross-modality Contrastive Representation Network Based on Rolling MLP Feature Extraction [J]. Computer Science, 2025, 52(11): 123-130.
[8] CHEN Chongyang, PENG Li, YANG Jielong. UAV Small Object Detection Algorithm Based on Feature Enhancement and Context Fusion [J]. Computer Science, 2025, 52(11): 131-140.
[9] DING Yuanbo, BAI Lin, LI Taoshen. Human-Object Interaction Detection Based on Fine-grained Attention Mechanism [J]. Computer Science, 2025, 52(11): 141-149.
[10] LI Xiaoyu, QIAN Yi, WEN Yimin, MIU Yuqing. Multi-level Feature Fusion Image Emotion Recognition Based on Color Enhancement [J]. Computer Science, 2025, 52(11): 157-165.
[11] WANG Yang, WANG Guodong, ZHAO Junli, SHENG Xiaomeng. Neural Radiance Field for Human Reconstruction Based on Multi-scale Hierarchical Network [J]. Computer Science, 2025, 52(11): 175-183.
[12] WANG Xiaofeng, HUANG Junjun, TAN Wenya, SHEN Zixuan. Object Detection Based on Deep Feature Enhancement and Path Aggregation Optimization [J]. Computer Science, 2025, 52(11): 184-195.
[13] GU Chengjie, MENG Yi, ZHU Dongjun, ZHANG Junjun. Medical Image Target Detection Method Based on Multi-branch Attention and Deep Down- sampling [J]. Computer Science, 2025, 52(11): 196-205.
[14] LIN Zukai, HOU Guojia, WANG Guodong, PAN Zhenkuan. Image Deraining Based on Union Attention Mechanism and Multi-stage Feature Extraction [J]. Computer Science, 2025, 52(11): 206-212.
[15] LIAO Jinchao, YANG Weizhe, QIN Yongbin, HUANG Ruizhang, CHEN Yanping, ZHOU Yulin. Method for Generating Judgment Documents Based on Trial Logic [J]. Computer Science, 2025, 52(11): 223-229.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!