Computer Science

Computer Science ›› 2024, Vol. 51 ›› Issue (5): 117-124.doi: 10.11896/jsjkx.230300049

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Multi Scale Progressive Transformer for Image Dehazing

ZHOU Yu1, CHEN Zhihua1, SHENG Bin2, LIANG Lei1   

  1. 1 College of Information Science and Engineering,East China University of Science and Technology,Shanghai 200237,China
    2 Department of Computer Science and Technology,Shanghai Jiao Tong University,Shanghai 200240,China
  • Received:2023-03-06 Revised:2023-06-28 Online:2024-05-15 Published:2024-05-08
  • About author:ZHOU Yu,born in 1994,Ph.D,is a member of CCF(No.D2304G).Her main research interests include compu-ter vision and image processing.
    CHEN Zhihua,born in 1969,Ph.D,professor,Ph.D supervisor,is a member of CCF(No.12441D).His main research interests include computer vision,machine learning,object detection and image video processing.
  • Supported by:
    National Natural Science Foundation of China(62272164) and Science and Technology on Space Intelligent Control Laboratory(HTKJ2022KL502010).

PDF (PC)

354

Abstract: In order to simultaneously recover image details and maintain global information in the dehazed image,a multi scale progressive transformer(MSP-Transformer) is proposed for image dehazing.The MSP-Transformer can effectively extract haze-related features from different scales,and restore clear image in a progressive way,achieving multi-scale learning and fusion of features and images.The proposed MSP-Transformer is divided into an encoding stage,a decoding stage,and a restoration stage.In the encoding stage,a Transformer block-based encoder is used to decompose the input image into different scales.The extracted haze-relevant features from different scales can fully characterize the information loss of the haze image.In the decoding stage,considering that different regions of the haze image have different information loss,this paper designs a feature aggregation module containing a multi-scale attention mechanism in decoder.The multi-scale attention contains channel attention and multi-scale spatial attention,and can fuse the feature information from different scales.The restoration stage contains restoration block and fusion block,firstly,the multi-scale feature fusion restoration block aggregates the haze relevant features from different scales to increase the association between these features,then the aggregated features are used to restore a haze-free image at each scale.Besides,the restored images from each scale are fused by fusion block to obtain the final dehazed result.Qualitative and quantitative experiments on both real and synthetic datasets show that the proposed MSP-Transformer has good dehazing performance.Compared with 11 state-of-the-art methods,MSP-Transformer obtains the best PSNR(39.53db) and SSIM(0.9954) on the RESIDE dataset,and achieves good visual effect.In addition,the ablation experiments also demonstrate the effectiveness of the proposed dehazing method.

Key words: Image dehazing, Multi scale, Transformer, Attention mechanism, Feature fusion

CLC Number: 

  • TP391
[1]HE K M,SUN J,TANG X.Single image haze removal usingdark channel prior[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2011,33(12):2341-2353.
[2]ZHU Q S,MAI J M,SHAO L.A Fast Single Image Haze Removal Algorithm Using Color Attenuation Prior[J].IEEE Transactions on Image Processing,2015,24(11):3522-3533.
[3]BERMAN D,TREIBITZ T,AVIDAN S.2020.Single Image Dehazing Using Haze-Lines[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,42(3):720-734.
[4]ZHANG J L,SHI D Y,JIA B.Insulator image defogging algorithm based on dark channel prior theory[J].Journal of Chongqing University of Technology(Natural Science),2022,36(7):208-215.
[5]QIN X,WANG Z L,BAI Y C,et al.FFA-Net:Feature Fusion Attention Network for Single Image Dehazing[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020,34:11908-11915.
[6]ZHANG J L,SHI D Y,JIA B.Insulator image defogging algorithm based on dark channel prior theory[J].Journal of Chongqing University of Technology(Natural Science),2022,36(7):208-215.
[7]LIU X H,MA Y R,SHI Z H,et al.GridDehazeNet:Attention-Based Multi-Scale Network for Image Dehazing[C]//Procee-dings of the IEEE/CVF International Conference on Computer Vision.2019:7313-7322.
[8]VASWANI A,SHAZEER N,PARMER N,et al.Attention is all you need[C]//Neural Information Processing Systems.2017:5998-6008.
[9]ZAMIR S W,ARORA A,KHAN S,et al.Restormer:Efficient Transformer for High-Resolution Image Restoration[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:5718-5729.
[10]WANG Z D,CUN X D,BAO J M,et al.Uformer:A GeneralU-Shaped Transformer for Image Restoration[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:17662-17672.
[11]SONG Y,HE Z Q,QIAN H,et al.2022.Vision Transformers for Single Image Dehazing[J].arXiv:2204.03883,2022.
[12]LIU Z,LIN Y T,CAO Y,et al.Swin transformer:Hierarchical vision transformer using shifted windows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:10012-10022.
[13]CAI B L,XU X M,JIA K,et al.Dehazenet:An end-to-end system for single image haze removal[J].IEEE Transactions on Image Processing,2016,25(11):5187-5198.
[14]REN W Q,LIU S,ZHANG H,et al.Single image dehazing via multi-scale convolutional neural networks[C]//Proceedings of the European Conference on Computer Vision.Springer,2016:154-169.
[15]WU H Y,QU Y Y,LIN S H,et al.Contrastive Learning for Compact Single Image Dehazing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:10551-10560.
[16]YU H,ZHENG N S,ZHOU M,et al.Frequency and SpatialDual Guidance for Image Dehazing[C]//Proceedings of the European Conference on Computer Vision.Springer,2022:181-198.
[17]SHAO Y J,LI L R H,REN W Q,et al.Domain Adaptation for Image Dehazing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:2805-2814.
[18]LI B Y,GOU Y B,GU S H,et al.You Only Look Yourself:Unsupervised and Untrained Single Image Dehazing Neural Network [J].International Journal of Computer Vision,2021,129(5):1754-1767.
[19]CHEN H T,WANG Y H,GUO T Y,et al.Pre-Trained Image Processing Transformer[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:12299-12310.
[20]LIANG J Y,CAO J Z,SUN G L,et al.SwinIR:Image Restoration Using Swin Transformer[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision Workshop.2021:1833-1844.
[21]LI X,JIN X,YU T,et al.Learning Omni-Frequency Region-adaptive Representations for Real Image Super-Resolution[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2021:1975-1983.
[22]LI X,WANG W H,HU X L,et al.Selective Kernel Networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:510-519.
[23]WANG X T,KELVIN C K C,YU K,et al.EDVR:Video Restoration With Enhanced Deformable Convolutional Networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.2019:1954-1963.
[24]LIU Y,PAN J S,REN J,et al.Learning Deep Priors for Image Dehazing[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019:2492-2500.
[25]DONG H,PAN J S,XIANG L,et al.Multi Scale Boosted Deha-zing Network With Dense Feature Fusion[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:2154-2164.
[26]HONG M,XIE Y,LI C H,et al.Distilling Image Dehazing With Heterogeneous Task Imitation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:3459-3468.
[27]CHEN Z Y,WANG Y C,YANG Y,et al.PSD:Principled Synthetic-to-Real Dehazing Guided by Physical Priors[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:7180-7189.
[28]ZHANG R,ISOLA P,EFROS A A,et al.The Unreasonable Effectiveness of Deep Features as a Perceptual Metric[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2018:586-595.
[29]MITTAL A,SOUNDARARAJAN R,BOVIK A C.Making a“Completely Blind” Image Quality Analyzer[J].IEEE Signal Processing Letters,2013,20(3):209-212.
[30]LI B Y,REN W Q,FU D P,et al.Benchmarking Single-Image Dehazing and Beyond [J].IEEE Transactions on Image Proces-sing,2010,28(1):492-505.
[31]YIN W,ZHANG J M,WANG O,et al.Learning To Recover 3D Scene Shape From a Single Image[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:204-213.
[1] ZHANG Jianliang, LI Yang, ZHU Qingshan, XUE Hongling, MA Junwei, ZHANG Lixia, BI Sheng. Substation Equipment Malfunction Alarm Algorithm Based on Dual-domain Sparse Transformer [J]. Computer Science, 2024, 51(5): 62-69.
[2] HE Shiyang, WANG Zhaohui, GONG Shengrong, ZHONG Shan. Cross-modal Information Filtering-based Networks for Visual Question Answering [J]. Computer Science, 2024, 51(5): 85-91.
[3] SHAN Xinxin, LI Kai, WEN Ying. Medical Image Segmentation Network Integrating Full-scale Feature Fusion and RNN with Attention [J]. Computer Science, 2024, 51(5): 100-107.
[4] WANG Ping, YU Zhenhuang, LU Lei. Partial Near-duplicate Video Detection Algorithm Based on Transformer Low-dimensionalCompact Coding [J]. Computer Science, 2024, 51(5): 108-116.
[5] BAI Xuefei, SHEN Wucheng, WANG Wenjian. Salient Object Detection Based on Feature Attention Purification [J]. Computer Science, 2024, 51(5): 125-133.
[6] WU Xiaoqin, ZHOU Wenjun, ZUO Chenglin, WANG Yifan, PENG Bo. Salient Object Detection Method Based on Multi-scale Visual Perception Feature Fusion [J]. Computer Science, 2024, 51(5): 143-150.
[7] LAN Yongqi, HE Xingxing, LI Yingfang, LI Tianrui. New Graph Reduction Representation and Graph Neural Network Model for Premise Selection [J]. Computer Science, 2024, 51(5): 193-199.
[8] HONG Tijing, LIU Dengfeng, LIU Yian. Radar Active Jamming Recognition Based on Multiscale Fully Convolutional Neural Network and GRU [J]. Computer Science, 2024, 51(5): 306-312.
[9] XI Ying, WU Xuemeng, CUI Xiaohui. Node Influence Ranking Model Based on Transformer [J]. Computer Science, 2024, 51(4): 106-116.
[10] WANG Ruiping, WU Shihong, ZHANG Meihang, WANG Xiaoping. Review of Vision-based Neural Network 3D Dynamic Gesture Recognition Methods [J]. Computer Science, 2024, 51(4): 193-208.
[11] XUE Jinqiang, WU Qin. Progressive Multi-stage Image Denoising Algorithm Combining Convolutional Neural Network and
Multi-layer Perceptron [J]. Computer Science, 2024, 51(4): 243-253.
[12] ZHANG Mingdao, ZHOU Xin, WU Xiaohong, QING Linbo, HE Xiaohai. Unified Fake News Detection Based on Semantic Expansion and HDGCN [J]. Computer Science, 2024, 51(4): 299-306.
[13] WANG Zihong, SHAO Yingxia, HE Jiyuan, LIU Jinbao. Sequential Recommendation Based on Multi-space Attribute Information Fusion [J]. Computer Science, 2024, 51(3): 102-108.
[14] HAO Ran, WANG Hongjun, LI Tianrui. Deep Neural Network Model for Transmission Line Defect Detection Based on Dual-branch Sequential Mixed Attention [J]. Computer Science, 2024, 51(3): 135-140.
[15] ZHANG Yang, XIA Ying. Object Detection Method with Multi-scale Feature Fusion for Remote Sensing Images [J]. Computer Science, 2024, 51(3): 165-173.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.