计算机科学 ›› 2023, Vol. 50 ›› Issue (6): 200-208.doi: 10.11896/jsjkx.220400288

• 计算机图形学&多媒体 • 上一篇    下一篇

基于动态卷积核的自适应图像去雾算法

刘哲, 梁宇栋, 李嘉莹   

  1. 山西大学计算机与信息技术学院 太原 030006
  • 收稿日期:2022-04-28 修回日期:2022-11-08 出版日期:2023-06-15 发布日期:2023-06-06
  • 通讯作者: 梁宇栋(liangyudong@sxu.edu.cn)
  • 作者简介:(202022407037@email.sxu.edu.cn)
  • 基金资助:
    国家自然科学基金(61802237,62272284,61906114);山西省研究生教育创新项目(2022Y127);山西省基础研究计划(自由探索类)项目(202203021221002,202203021211291);山西省自然科学基金(201901D211176,201901D211170,202103021223464);山西省高等学校科技创新项目(2019L0066);山西省科技重大专项计划(202101020101019);山西省重点研发计划(国际科技合作,201903D421041,能源与节能环保领域,202102070301019);山西省科技创新人才团队专项资助

Adaptive Image Dehazing Algorithm Based on Dynamic Convolution Kernels

LIU Zhe, LIANG Yudong, LI Jiaying   

  1. School of Computer and Information Technology,Shanxi University,Taiyuan 030006,China
  • Received:2022-04-28 Revised:2022-11-08 Online:2023-06-15 Published:2023-06-06
  • About author:LIU Zhe,born in 1998,master.His main research interests includes computer vision and image processing.LIANG Yudong,born in 1988,associate professor.His main research interests includes computer vision,image processing,and deep learning based applications.
  • Supported by:
    National Natural Science Foundation of China(61802237,62272284,61906114),Graduate Education Innovation Programs of Shanxi Province(2022Y127),Fundamental Research Program of Shanxi Province(202203021221002,202203021211291),Natural Science Foundation of Shanxi Province(201901D211176,201901D211170,202103021223464) ,Scientific and Technological Innovation Programs of Higher Education Institutions in Shanxi(2019L0066),Science and Technology Major Project of Shanxi Province (202101020101019),Key R & D Program of Shanxi Province(201903D421041,202102070301019) and Special Fund for Science and Technology Innovation Teams of Shanxi Province.

摘要: 现有图像去雾方法普遍存在去雾不彻底、容易出现颜色失真等问题,基于传统深度学习模型的图像去雾方法多采用静态推理模式,在该模式下,模型对不同样本会采用同样的、固定的参数设置,从而抑制了模型的表达能力,影响图像的去雾效果。针对以上问题,文中提出了一种基于动态卷积核的自适应图像去雾算法,该算法包括编码网络、自适应特征增强网络和解码网络3个部分。文中采用动态卷积、密集残差、注意力机制设计了自适应特征增强网络,该网络主要包括动态残差组件和动态跨层特征融合组件。动态残差组件由动态密集残差模块、一个卷积层和双注意力模块构成,其中动态密集残差模块将动态卷积引入密集残差模块,同时设计了一个基于注意力的权重动态聚合子网络,动态地生成卷积核参数以达到样本自适应的目的,在减少信息丢失的同时增强了模型的表达能力;双注意力模块结合通道注意力和像素注意力,使模型更加关注图像通道之间的差异性以及雾霾分布不均匀的区域。动态跨层特征融合组件通过动态融合不同阶段的特征,来学习丰富的上下文信息,防止网络深层计算时遗忘网络的早期特征,同时极大地丰富了特征表示,有利于模型对无雾图像细节信息的恢复。在合成数据集和真实数据集上进行了大量实验,结果表明,所提方法不仅取得了较好的客观评价分数,而且重建了主观效果较好的去雾图像,超越了对比方法的性能。

关键词: 图像去雾, 深度学习, 动态神经网络, 注意力机制, 特征融合

Abstract: Existing image dehazing methods generally have problems such as incomplete dehazing and color distortion.Image dehazing methods based on traditional deep learning models mostly use static inference during testing,which use the same and fixed parameters for different samples,thereby inhibiting the expressive ability of the model and decreasing the dehazing performance.Aiming at the above problems,this paper proposes an adaptive image dehazing algorithm based on dynamic convolution kernel.The proposed model includes three parts:encoding network,adaptive feature enhancement network and decoding network.This paper combines dynamic convolutions,dense residual connections,and attention mechanism to complete the adaptive feature enhancement network,which mainly includes dynamic residual components and dynamic skip-connected feature fusion components.The dynamic residual component is composed of a dynamic residual dense block,a convolutional layer and a dual attention mo-dule.The dynamic residual dense block introduces dynamic convolutions into the residual dense block,and an attention-based weight dynamic aggregator is designed at the same time,which dynamically generates adaptive convolution kernel parameters.The dynamic convolutions have reduced the loss of information and enhanced the expressive ability of the model.The dual attention module combines channel attention and pixel attention to make the model pay more attention to the differences between image channels and areas with uneven distribution of haze.The dynamic skip-connected feature fusion component learns rich contextual information by dynamically fusing the features of different stages via skip-connections,preventing the early features of the network from being forgotten when the information flows into deeper layers.Meanwhile,the feature representations are greatly enriched,which benefits the restorations of the details for fog-free images.Extensive experiments on synthetic datasets and real datasets show that our method not only achieves better objective evaluation scores,but also reconstructs dehazing images with better visual effects,surpassing the performance of compared methods.

Key words: Image dehazing, Deep learning, Dynamic neural network, Attention mechanism, Feature fusion

中图分类号: 

  • TP391
[1]KHAN M F,KHAN E,ABBASI Z A.Segment dependent dynamic multi-histogram equalization for image contrast enhancement[J].Digital Signal Processing,2014,25:198-223.
[2]LI H,XIE W,WANG X,et al.GPU Implementation ofMulti-scale Retinex Image Enhancement Algorithm[C]//2016 IEEE/ACS 13thInternational Conference of Computer Systems and APpplications(AICCSA).IEEE,2016:1-5.
[3]HE K,SUN J,TANG X.Single image haze removal using dark channel prior[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2010,33(12):2341-2353.
[4]CAI B,XU X,JIA K,et al.Dehazenet:An end-to-end system for single image haze removal[J].IEEE Transactions on Image Processing,2016,25(11):5187-5198.
[5]REN W,LIU S,ZHANG H,et al.Single image dehazing viamulti-scale convolutional neural networks[C]//European Conference on Computer Vision.Cham:Springer,2016:154-169.
[6]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[7]ZHANG Y,TIAN Y,KONG Y,et al.Residual dense network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:2472-2481.
[8]ARIGELA S,ASARI V K.Enhancement of hazy color imagesusing a self-tunable transformation function[C]//International Symposium on Visual Computing.Cham:Springer,2014:578-587.
[9]BERMAN D,AVIDAN S.Non-local image dehazing[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:1674-1682.
[10]REN W,MA L,ZHANG J,et al.Gated fusion network for single image dehazing[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:3253-3261.
[11]LI B,PENG X,WANG Z,et al.Aod-net:All-in-one dehazing network[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:4770-4778.
[12]ZHANG H,PATEL V M.Densely connected pyramid dehazing network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:3194-3203.
[13]YI Q,LI J,FANG F,et al.Efficient and accurate multi-scale topological network for single image dehazing[J].IEEE Transactions on Multimedia,2021,24:3114-3128.
[14]ZHANG X,JIANG R,WANG T,et al.Single image dehazing via dual-path recurrent network[J].IEEE Transactions on Image Processing,2021,30:5211-5222.
[15]WU H,QU Y,LIN S,et al.Contrastive learning for compactsingle image dehazing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:10551-10560.
[16]YANG Y,WANG C,LIU R,et al.Self-Augmented Unpaired Image Dehazing via Density and Depth Decomposition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:2037-2046.
[17]LIU H,WU Z,LI L,et al.Towards Multi-Domain Single Image Dehazing via Test-Time Training[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:5831-5840.
[18]WU Z,NAGARAJAN T,KUMAR A,et al.Blockdrop:Dynamic inference paths in residual networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:8817-8826.
[19]YANG L,HAN Y,CHEN X,et al.Resolution adaptive net-works for efficient inference[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:2369-2378.
[20]YANG B,BENDER G,LE Q V,et al.Condconv:Conditionally parameterized convolutions for efficient inference[J].Advances in Neural Information Processing Systems,2019,32:1-11.
[21]MA N,ZHANG X,HUANG J,et al.Weightnet:Revisiting the design space of weight networks[C]//European Conference on Computer Vision.Springer,Cham,2020:776-792.
[22] CHEN Y,DAI X,LIU M,et al.Dynamic convolution:Attention over convolution kernels[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:11030-11039.
[23]HU J,SHEN L,SUN G.Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:7132-7141.
[24]WOO S,PARK J,LEE J Y,et al.Cbam:Convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision(ECCV).2018:3-19.
[25]DOSOVITSKIY A,BEYER L,KOLESNIKOV A,et al.Animage is worth 16x16 words:Transformers for image recognition at scale[J].arXiv:2010.11929,2020.
[26]GUO M H,XU T X,LIU J J,et al.Attention Mechanisms in Computer Vision:A Survey[J].arXiv:2111.07624,2021.
[27]LIU X,MA Y,SHI Z,et al.Griddehazenet:Attention-basedmulti-scale network for image dehazing[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019:7314-7323.
[28]HONG M,XIE Y,LI C,et al.Distilling image dehazing with heterogeneous task imitation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:3462-3471.
[29]QIN X,WANG Z,BAI Y,et al.FFA-Net:Feature fusion attention network for single image dehazing[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:11908-11915.
[30]LI B,REN W,FU D,et al.Benchmarking single-image dehazing and beyond[J].IEEE Transactions on Image Processing,2018,28(1):492-505.
[31]ANCUTI C O,ANCUTI C,TIMOFTE R.NH-HAZE:Animage dehazing benchmark with non-homogeneous hazy and haze-free images[C]//Proceedings of the IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition Workshops.2020:444-445.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!