计算机科学 ›› 2024, Vol. 51 ›› Issue (6A): 230600039-6.doi: 10.11896/jsjkx.230600039

• 图像处理&多媒体技术 • 上一篇    下一篇

基于注意力机制的残差特征聚合网络超分辨率图像重建研究

孙阳, 丁建伟, 张琪, 魏慧雯, 田博文   

  1. 中国人民公安大学信息网络安全学院 北京 100038
  • 发布日期:2024-06-06
  • 通讯作者: 丁建伟(jwding@ppsuc.edu.cn)
  • 基金资助:
    中国人民公安大学安全防范工程双一流专项(2023SYL08)

Study on Super-resolution Image Reconstruction Using Residual Feature Aggregation NetworkBased on Attention Mechanism

SUN Yang, DING Jianwei, ZHANG Qi, WEI Huiwen, TIAN Bowen   

  1. School of Information Network Security,People’s Public Security University of China,Beijing 100038,China
  • Published:2024-06-06
  • About author:SUN Yang,born in 1998,postgraduate.His main research interests include image super-resolution and computer vision.
    DING Jianwei,born in 1984,Ph.D,associate professor.His main research in-terests include computer vision and artificial intelligence security.
  • Supported by:
    People’s Public Security University of China Double First-class Project on Security and Prevention Engineering(2023SYL08).

摘要: 针对单图像超分辨率算法级联残差块的输出特征仅在局部作用的问题,提出了一种结合注意力机制的残差特征聚合网络。该网络通过跳跃连接将各残差块输出不同层次的特征聚合到残差组的尾部,实现特征的充分提取与复用,扩大网络的感受野并增强特征的表达能力,使得不同层次的特征图更充分地参与到图像重建中。同时,为增强特征信息空间上的相关性,引入增强空间注意力机制以改善残差块的性能。大量实验表明,此模型可以获得良好的超分辨率性能。在×4倍SR任务中与RCAN,SAN和HAN等主流方法相比,在5个基准测试集上取得的峰值信噪比平均提升0.07dB,0.06dB,0.006dB,结构相似度平均提升0.001 2,0.001 1,0.0008,重建图像质量明显提高,细节更加丰富,充分说明了所提方法的有效性与先进性。

关键词: 图像超分辨率重建, 深度学习, 注意力机制, 特征聚合, 卷积神经网络

Abstract: To address the problem of the local effect of the output features of cascaded residual blocks in single image super-resolution algorithms,a residual feature aggregation network combined with attention mechanism is proposed.The network aggregates the features of different levels output by each residual block through skip connections to the end of the residual group,achieves sufficient feature extraction and reuse,expands the receptive field of the network and enhances the expression ability of features.Meanwhile,to improve the spatial correlation of feature information,an enhanced spatial attention mechanism is introduced to improve the performance of the residual blocks.Extensive experiments demonstrate that the proposed model achieves good super-resolution performance.Compared with state-of-the-art methods such as RCAN,SAN,and HAN,the proposed method demonstrates significant effectiveness and advancement in the task of ×4 super-resolution.On five benchmark datasets,our method achieves an average improvement of 0.07dB,0.06dB,and 0.006dB in peak signal-to-noise ratio,as well as an average improvement of 0.001 2,0.001 1,and 0.0008 in structural similarity index.The reconstructed images exhibit a notable increase in quality,with more abundant details.These results verifies he efficacy and advancement of the proposed method.

Key words: Image super-resolution reconstruction, Deep learning, Attention mechanismsl, Feature aggregation, Convolutional neural networks

中图分类号: 

  • TP391.4
[1]ZHANG L,WU X.An edge-guided image interpolation algorithm via directional filtering and data fusion[J].IEEE Transactions on Image Processing,2006,15(8):2226-2238.
[2]ZHANG K,GAO X,TAO D,et al.Single image super-resolution with non-local means and steering kernel regression[J].IEEE Transactions on Image Processing,2012,21(11):4544-4556.
[3]DONG C,LOY C C,HE K,et al.Learning a deep convolutional network for image super-resolution[C]//Computer Vision-ECCV 2014:13th European Conference.2014:184-199.
[4]KIM J,LEE J K,LEE K M.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:1646-1654.
[5]LEDIG C,THEIS L,HUSZÁR F,et al.Photo-realistic single image super-resolution using a generative adversarial network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:4681-4690.
[6]LIM B,SON S,KIM H,et al.Enhanced deep residual networks for single image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops.2017:136-144.
[7]ZHANG Y,LI K,LI K,et al.Image super-resolution using very deep residual channel attention networks[C]//Proceedings of the European Conference on Computer Vision(ECCV).2018:286-301.
[8]SHI W,CABALLERO J,HUSZÁR F,et al.Real-time single im-age and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:1874-1883.
[9]HU J,SHEN L,SUN G.Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:7132-7141.
[10]DAI T,CAI J,ZHANG Y,et al.Second-order Attention Network for Single Image Super-Resolution[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2019.
[11]NIU B,WEN W,REN W,et al.Single image super-resolution via a holistic attention network[C]//Computer Vision-ECCV 2020:16th European Conference.2020:191-207.
[12]LAI W S,HUANG J B,AHUJA N,et al.Deep laplacian pyramid networks for fast and accurate super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:624-632.
[13]DONG C,LOY C C,HE K,et al.Image super-resolution using deep convolutional networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,38(2):295-307.
[14]JOHNSON J,ALAHI A,FEI-FEI L.Perceptual losses for real-time style transfer and super-resolution[C]//Computer Vision-ECCV 2016:14th European Conference.2016:694-711.
[15]GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al.Generative Adversarial Nets[J].arXiv:1406.2661,2014.
[16]HE K,ZHANG X,REN S,et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[17]ZHANG Y,TIAN Y,KONG Y,et al.Residual dense networkfor image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:2472-2481.
[18]LI J,FANG F,MEI K,et al.Multi-scale residual network forimage super-resolution[C]// Proceedings of the European Conference on Computer Vision(ECCV).2018:517-532.
[19]LIU J,ZHANG W,TANG Y,et al.Residual feature aggregation network for image super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:2359-2368.
[20]WANG X,YU K,WU S,et al.Esrgan:Enhanced super-resolution generative adversarial networks[C]//Proceedings of the European Conference on Computer Vision(ECCV) Workshops.2018.
[21]LIU J,TANG J,WU G.Residual feature distillation network for lightweight image super-resolution[C]//Computer Vision-ECCV 2020 Workshops.2020:41-55.
[22]AGUSTSSON E,TIMOFTE R.NTIRE 2017 Challenge on Single Image Super-Resolution:Dataset and Study[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops(CVPRW).IEEE,2017.
[23]HARIS M,SHAKHNAROVICH G,UKITA N.Deep back-projection networks for super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:1664-1673.
[24]YANG X,ZHU Y,GUO Y,et al.An image super-resolutionnetwork based on multi-scale convolution fusion[J].The Visual Computer,2022,38(12):4307-4317.
[25]HAO F,ZHANG T,ZHAO L,et al.Efficient residual attention network for single image super-resolution[J].Applied Intelligence,2022,52(1):652-661.
[26]LIU Z X,ZHU C J,HUANG J,et al.lmage Super-resolution by Residual Attention Network with Multi-skip Connection[J].Chinese Journal of Computers,2021,48(11):258-267.
[27]GUO L,LI C,CHEN C,et al.Image Super-Resolution Recon-struction Using Recursive Residual Network Based onChannel Attention[J].Chinese Journal of Computers,2021,48(8):139-144.
[28]LIU Z X,ZHU C J,HUANG J et al.lmage Super-resolution by Residual Attention Network with Multi-skip Connection[J].Chinese Journal of Computers,2021,48(11):258-267.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!