基于动态金字塔和子空间注意力的图像超分辨率重建网络

doi:10.11896/jsjkx.210900202

摘要/Abstract

摘要： 针对现有单图像超分辨率卷积神经网络存在模型参数过多以及重建失真过大的问题,提出了一种基于动态金字塔结构与子空间注意力模块的轻量级单图像超分辨率网络模型。首先,所采用的动态多尺度金字塔特征组合模块的网络主体由动态卷积和金字塔分组卷积构成。其次,动态卷积可以根据不同的图像内容自适应地进行不同的卷积操作,从而对不同的图像提取出不同的特征;金字塔分组卷积不仅可以更好地提取多尺度图像特征信息,而且能够有效降低网络模型的参数量。最后,在网络模型末端采用子空间注意力模块,将图像的通道空间分为多个子空间,并为每个子空间学习不同的注意力图,这样不仅可以更好地捕获图像的跨通道相关信息,而且可以有效融合各子空间的图像特征信息。与现有主流算法相比,所提方法不仅具有更小的网络模型参数量,而且重建出的超分辨率图像在视觉效果和定量分析方面均能取得更好的表现。

关键词: 超分辨率, 轻量级, 动态卷积, 金字塔分组卷积, 子空间注意力模块

Abstract: Aiming at the problems of excessive model parameters and reconstruction distortion in existing single-image super-reso-lution convolutional neural networks,a lightweight single-image super-resolution network model based on dynamic pyramid structure and subspace attention module is proposed.First,the network body with dynamic multi-scale pyramid feature combination module consists of dynamic convolution and pyramid grouping convolution.Dynamic convolution can adaptively perform different convolution operations for different images,so as to extract different features for different images.Pyramid grouping convolution not only can better extract multi-scale features,but also can effectively reduce the number of parameters of the network model.Finally,a subspace attention module is used at the end of the network model to divide the channel space of images into multiple subspaces and learn different attention maps for each subspace,which not only can better capture the cross-channel rela-ted information of images,but also allows for effective fusion of image feature information of each subspace.Compared with the existing mainstream algorithms,the proposed method not only has a smaller number of model parameters,but also the reconstructed super-resolution images can achieve better performance in terms of visual effects and quantitative analysis.

Key words: Super-resolution, Lightweight, Dynamic convolution, Pyramid grouping convolution, Subspace attention block

中图分类号:

TP391

何鹏浩, 余映, 徐超越. 基于动态金字塔和子空间注意力的图像超分辨率重建网络[J]. 计算机科学, 2022, 49(11A): 210900202-8. https://doi.org/10.11896/jsjkx.210900202

HE Peng-hao, YU Ying, XU Chao-yue. Image Super-resolution Reconstruction Network Based on Dynamic Pyramid and Subspace Attention[J]. Computer Science, 2022, 49(11A): 210900202-8. https://doi.org/10.11896/jsjkx.210900202

参考文献

[1]FREEMAN W T,PAZSTOR E C,CARMICHAEL O T.Lear-ning low-level vision[J].International Journal of Computer Vision,2000,40(1):25-47.
[2]ZHANG Y,FAN Q,BAO F,et al.Single-image super-resolution based on rational fractal interpolation[J].IEEE Transactions on Image Processing,2018,27(8):3782-3797.
[3]YANG M C,WANG Y C F.A self-learning approach to single image super-resolution[J].IEEE Transactions on Multimedia,2012,15(3):498-508.
[4]DONG C,LOY C C,HE K,et al.Learning a deep convolutional network for image super-resolution[C]//European Conference on Computer Vision.Springer,2014:184-199.
[5]DONG C,LOY C C,TANG X.Accelerating the super-resolution convolutional neural network[C]//European Conference on Computer Vision.Springer,2016:391-407.
[6]LIM B,SON S,KIM H,et al.Enhanced deep residual networks for single image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops.IEEE,2017:136-144.
[7]TONG T,LI G,LIU X,et al.Image super-resolution usingdense skip connections[C]//Proceedings of the IEEE International Conference on Computer Vision.IEEE,2017:4799-4807.
[8]ZHANG Y,TIAN Y,KONG Y,et al.Residual dense network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2018:2472-2481.
[9]HUANG G,LIU Z,VAN DER MAATEN L,et al.Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2017:4700-4708.
[10]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2016:770-778.
[11]ZHANG Y L,LI K P,LI K,et al.Image super-resolution using very deep residual channel attention networks[C]//Proceedings of the European Conference on Computer Vision(ECCV).Springer,2018:286-301.
[12]LIU J,ZHANG W,TANG Y,et al.Residual feature aggregation network for image super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.IEEE,2020:2359-2368.
[13]ANWAR S,BARNES N.Densely residuallaplacian super-resolution[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,1(1):1-1.
[14]LI J,FANG F,MEI K,et al.Multi-scale residual network forimage super-resolution[C]//Proceedings of the European Conference on Computer Vision(ECCV).Springer,2018:517-532.
[15]KIM J,LEE J K,LEE K M.Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2016:1637-1645.
[16]TAI Y,YANG J,LIU X.Image super-resolution via deep recursive residual network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2017:3147-3155.
[17]AHN N,KANG B,SOHN K A.Fast,accurate,and lightweight super-resolution with cascading residual network[C]//Procee-dings of the European Conference on Computer Vision(ECCV).Springer,2018:252-268.
[18]LAI W S,HUANG J B,AHUJA N,et al.Deeplaplacian pyramid networks for fast and accurate super-resolution[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2017:624-632.
[19]HUI Z,GAO X,YANG Y,et al.Lightweight image super-resolution with information multi-distillation network[C]//Procee-dings of the 27th ACM International Conference on Multimedia.ACM,2019:2024-2032.
[20]WOO S,PARK J,LEE J Y,et al.Cbam:Convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision(ECCV).Springer,2018:3-19.
[21]BOUREAU Y L,BACH F,LECUN Y,et al.Learning mid-level features for recognition[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.IEEE,2010:2559-2566.
[22]AGUSTSSON E,TIMOFTE R.Ntire 2017 challenge on single image super-resolution:Dataset and study[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops.IEEE,2017:126-135.
[23]BEVILACQUA M,ROUMY A,GUILLEMOT C,et al.Low-Complexity Single-Image Super-Resolution based on Nonnegative Neighbor Embedding[C]//Proceedings of the 23rd British Machine Vision Conference.BMVA Press,2012:135.1-135.10.
[24]ZEYDE R,ELAD M,PROTTER M.On single image scale-up using sparse-representations[C]//International Conference on Curves and Surfaces.Springer,2010:711-730.
[25]MARTIN D,FOWLKES C,TAL D,et al.A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//Proceedings of the Eighth IEEE International Conference on Computer Vision.IEEE,2001:416-423.
[26]HUANG J B,SINGH A,AHUJA N.Single image super-resolution from transformed self-exemplars[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2015:5197-5206.
[27]KIM J,KWON LEE J,MU LEE K.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2016:1646-1654.
[28]TAI Y,YANG J,LIU X,et al.A persistent memory network for image restoration[C]//Proceedings of the IEEE International Conference on Computer Vision.IEEE,2017:4549-4557.
[29]ZHANG K,ZUO W,ZHANG L.Learning a single convolutional super-resolution network for multiple degradations[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2018:3262-3271.
[30]LI Z,YANG J,LIU Z,et al.Feedback network for image super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.IEEE,2019:3867-3876.
[31]ZHU F,ZHAO Q.Efficient single image super-resolution viahybrid residual feature learning with compact back-projection network[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops.IEEE,2019.
[32]LI J X,HUANG Z Y,LI W B,et al.Image super-resolution based on multi-level feature fusion[J/OL].Acta Automatica Sinica,2021:1-11.http://www.aas.net.cn/cn/article/doi/10.16383/j.aas.c200585.

相关文章 15

[1]	戴朝霞, 李锦欣, 张向东, 徐旭, 梅林, 张亮. 基于DNGAN的磁共振图像超分辨率重建算法 Super-resolution Reconstruction of MRI Based on DNGAN 计算机科学, 2022, 49(7): 113-119. https://doi.org/10.11896/jsjkx.210600105
[2]	沈超, 何希平. 基于纹理特征增强和轻量级网络的人脸防伪算法 Face Anti-spoofing Algorithm Based on Texture Feature Enhancement and Light Neural Network 计算机科学, 2022, 49(6A): 390-396. https://doi.org/10.11896/jsjkx.210600217
[3]	周颖, 常明新, 叶红, 张燕. 基于元迁移的太阳能电池板缺陷图像超分辨率重建方法 Super Resolution Reconstruction Method of Solar Panel Defect Images Based on Meta-transfer 计算机科学, 2022, 49(3): 185-191. https://doi.org/10.11896/jsjkx.210100234
[4]	陈贵强, 何军. 自然场景下遥感图像超分辨率重建算法研究 Study on Super-resolution Reconstruction Algorithm of Remote Sensing Images in Natural Scene 计算机科学, 2022, 49(2): 116-122. https://doi.org/10.11896/jsjkx.210700095
[5]	冷佳旭, 王佳, 莫梦竟成, 陈泰岳, 高新波. 基于深度学习的视频超分辨率重构进展综述 Survey on Video Super-resolution Based on Deep Learning 计算机科学, 2022, 49(2): 123-133. https://doi.org/10.11896/jsjkx.211000007
[6]	周文辉, 石敏, 朱登明, 周军. 基于残差注意力网络的地震数据超分辨率方法 Seismic Data Super-resolution Method Based on Residual Attention Network 计算机科学, 2021, 48(8): 24-31. https://doi.org/10.11896/jsjkx.200900034
[7]	龚浩田, 张萌. 基于关键点检测的无锚框轻量级目标检测算法 Lightweight Anchor-free Object Detection Algorithm Based on Keypoint Detection 计算机科学, 2021, 48(8): 106-110. https://doi.org/10.11896/jsjkx.200700161
[8]	郭琳, 李晨, 陈晨, 赵睿, 范仕霖, 徐星雨. 基于通道注意递归残差网络的图像超分辨率重建 Image Super-resolution Reconstruction Using Recursive ResidualNetwork Based on ChannelAttention 计算机科学, 2021, 48(8): 139-144. https://doi.org/10.11896/jsjkx.200500150
[9]	程松盛, 潘金山. 基于深度学习特征匹配的视频超分辨率方法 Video Super-resolution Method Based on Deep Learning Feature Warping 计算机科学, 2021, 48(7): 184-189. https://doi.org/10.11896/jsjkx.200800224
[10]	王伟, 胡涛, 李欣蔚, 沈思婉, 姜小明, 刘峻源. 白细胞图像超分辨率重建研究 Study on Super-resolution Image Reconstruction of Leukocytes 计算机科学, 2021, 48(4): 164-168. https://doi.org/10.11896/jsjkx.200100099
[11]	许虞俊, 李晨. 基于YOLO优化的轻量级目标检测网络 Light-weight Object Detection Network Optimized Based on YOLO Family 计算机科学, 2021, 48(11A): 265-269. https://doi.org/10.11896/jsjkx.201000152
[12]	陆要要, 袁家斌, 何珊, 王天星. 基于超分辨率重建的低质量视频人脸识别方法 Low-quality Video Face Recognition Method Based on Super-resolution Reconstruction 计算机科学, 2021, 48(11A): 295-302. https://doi.org/10.11896/jsjkx.201200159
[13]	刘彦, 秦品乐, 曾建朝. 基于YOLOv3与分层数据关联的多目标跟踪算法 Multi-object Tracking Algorithm Based on YOLOv3 and Hierarchical Data Association 计算机科学, 2021, 48(11A): 370-375. https://doi.org/10.11896/jsjkx.201000115
[14]	谢海平, 李高源, 杨海涛, 赵洪利. 超分辨率重构遥感图像分类研究 Classification Research of Remote Sensing Image Based on Super Resolution Reconstruction 计算机科学, 2021, 48(11A): 424-428. https://doi.org/10.11896/jsjkx.210300132
[15]	王瑞锦, 唐榆程, 裴锡凯, 郭上铜, 张凤荔. 基于轻量级同态加密和零知识证明的区块链隐私保护方案 Block-chain Privacy Protection Scheme Based on Lightweight Homomorphic Encryption and Zero-knowledge Proof 计算机科学, 2021, 48(11A): 547-551. https://doi.org/10.11896/jsjkx.201200138

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed