计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (2): 205-216.doi: 10.11896/jsjkx.230800017

• 计算机图形学&多媒体 • 上一篇    下一篇

基于递归门控卷积的遥感图像超分辨率研究

刘长新1, 吴宁2, 胡俐蕊3, 高霸1, 高学山4   

  1. 1 广西大学计算机与电子信息学院 南宁530004
    2 广西海洋工程装备与技术重点实验室(北部湾大学) 广西 钦州535011
    3 北部湾大学电子与信息工程学院 广西 钦州535011
    4 北部湾大学机械与船舶海洋工程学院 广西 钦州535011
  • 收稿日期:2023-08-04 修回日期:2023-11-27 出版日期:2024-02-15 发布日期:2024-02-22
  • 通讯作者: 胡俐蕊(hulr163@163.com)
  • 作者简介:(m18278591041@163.com)
  • 基金资助:
    国家自然科学基金(61961004);广西重点研发计划(2021AB10030)

Recursive Gated Convolution Based Super-resolution Network for Remote Sensing Images

LIU Changxin1, WU Ning2, HU Lirui3, GAO Ba1, GAO Xueshan4   

  1. 1 School of Computer,Electronics and Information,Guangxi University,Nanning,530004,China
    2 Guangxi Key Laboratory of Marine Engineering Equipment and Technology,Beibu Gulf University,Qinzhou,Guangxi 535011,China
    3 College of Electronics and Information Engineering,Beibu Gulf University,Qinzhou,Guangxi 535011,China
    4 College of Mechanical,Naval Architecture and Ocean Engineering,Beibu Gulf University,Qinzhou,Guangxi 535011,China
  • Received:2023-08-04 Revised:2023-11-27 Online:2024-02-15 Published:2024-02-22
  • About author:LIU Changxin,born in 1998,postgra-duate.His main research interests include image processing and deep lear-ning.HU Lirui,born in 1966,Ph.D,professor,master supervisor.His main research interests include embedded system,image recognition,and machine learning.
  • Supported by:
    National Natural Science Foundation of China(61961004) and Guangxi Key Project of Research and Development Plan(2021AB10030).

PDF (PC)

1721

摘要： 由于受到硬件条件的限制,通常难以获得具有高分辨率(HR)的遥感图像。利用单幅图像超分辨率(SISR)技术对低分辨率(LR)遥感图像进行超分辨率重建是获取高分辨率遥感图像的常用方法。近年来,在图像超分辨率领域引入的卷积神经网络(CNN)改进了图像重建性能。然而,现有的基于CNN的超分辨率模型通常使用低阶注意力机制提取深层特征,其表征能力有待提高,且常规卷积的感受野有限,缺乏对远距离依赖关系的学习。为了解决以上问题,提出了一种基于递归门控卷积的遥感图像超分辨率方法RGCSR。该方法引入递归门控卷积gnConv学习全局依赖和局部细节,通过高阶空间交互来获取高阶特征。首先,使用由高阶交互子模块(HorBlock)和前馈神经网络(FFN)组成的高阶交互——前馈神经网络模块(HFB)提取高阶特征。其次,利用由通道注意力(CA)和gnConv构建的特征优化模块(FOB)优化各个中间模块的输出特征。最后,在多个数据集上的对比结果表明,RGCSR比现有的基于CNN的超分辨率方法具备更好的重建性能和视觉效果。

关键词: 递归门控卷积, 高阶空间交互, 通道注意力, 遥感图像, 超分辨率

Abstract: Due to hardware manufacturing constraints,it is usually difficult to obtain high-resolution(HR) images in the area of remote sensing.From low resolution remote-sensing image to reconstruct high-resolution(HR) image via single-image super-re-solution(SISR) technique is a common method.Recently,the convolutional neural network(CNN) was introduced to the field of super-resolution image reconstruction,and it effectively improved the image reconstruction performance.However,the classic CNN-based approaches typically use low-order attention to extract deep features,which limites its reconstructing ability.More-over,the receptive field is limited,which lacks the ability to learn long-range dependency.To solve the above problems,a recursive gated convolution-based super-resolution method for remote sensing images(RGCSR) is proposed.The RGCSR introduces recursive gated convolution(gnConv) to learn global dependencies and local details,and high-order features are acquired by high-order spatial interactions.Firstly,a high-order interaction—feedforward neural network(HFB) consisting of a high-order interaction sub-module(HorBlock) and a feedforward neural network(FFN) is applied to extract high-order features.Then,a feature optimization module(FOB) contains channel attention(CA) and gnConv is used to optimize the output features of each intermediate module.Finally,the comparison results on multiple datasets show that RGCSR has better reconstruction and visualization performances than existing CNN-based solutions.

Key words: Recursive gated convolution, High-order spatial interaction, Channel attention, Remote sensing images, Super-resolution

中图分类号: 

  • TP751.1
[1]SONG W,GU H H,SONG W,et al.Environmental assessments in dense mining areas using remote sensing information over qian’an and qianxi regions china[J].Ecological Indicators,2023,146:1-13.
[2]GARCÍA-PARDO K A,MORENO-RANGEL D,DOMÍNGUEZ-AMARILLO S,et al.Remote sensing for the assessment of ecosystem services provided by urban vegetation.a review of the methods applied[J].Urban Forestry & Urban Greening,2022,74:1-12.
[3]XING Z,YANG S,ZAN X,et al.Flood vulnerability assessment of urban buildings based on integrating high-resolution remote sensing and street view images[J].Sustainable Cities and Society,2023,92:1-14.
[4]CHENG L B,LI X Y,LI Z,et al.Remote sensing image denoi-sing method based on curvelet transform and goodness-of-fit test[J].Journal of Jilin University(Engineering and Technology Edition),2023,53(11):3207-3213.
[5]WU Y Y,WANG Y J,XIE Y Y.The panchromatic sharpening method based on multi-scale residual attention network[J].Journal of Chongqing University of Technology(Natural Science),2023,37(8):222-230.
[6]ZHANG D Y,SHAO J,LI X Y,et al.Remote sensing image super-resolution via mixed high-order attention network[J].IEEE Transactions on Geoscience and Remote Sensing,2021,59(6):5183-5196.
[7]ISAAC J S,KULKARNI R.Super resolution techniques formedical image processing[C]//International Conference on Technologies for Sustainable Development.Mumbai:IEEE,2015:1-6.
[8]RASTI P,UIBOUPIN T,ESCALERA S,et al.Convolutionalneural network super resolution for face recognition in surveillance monitoring[C]//Articulated Motion and Deformable Objects:9th International Conference.Palma de Mallorca:Sprin-ger,2016:175-184.
[9]DONG C,LOY C C,HE K M,et al.Learning a deep convolutional network for image super-resolution[C]//Proceedings of the European Conference on Computer Vision(ECCV).Mumbai:Springer,2014:184-199.
[10]DUCHON C E.Lanczos Filtering in One and Two Dimensions[J].Journal of Applied Meteorology and Climatology,1979,18(8):1016-1022.
[11]SUN J,XU Z B,SHUM H Y,et al.Image super-resolution using gradient profile prior[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Ancho-rage:IEEE,2008:1-8.
[12]KIM J,LEE J K,LEE K M.Accurate image super-resolutionusing very deep convolutional networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Anchorage:IEEE,2016:1646-1654.
[13]LIM B,SON S,KIM H,et al.Enhanced deep residual networks for single image super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.Honolulu:IEEE,2017:136-144.
[14]HARIS M,SHAKHNAROVICH G,UKITA N.Deep back-projection networks for super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City:IEEE,2018:1664-1673.
[15]ZHANG Y L,TIAN Y P,KONG Y,et al.Residual dense network for image super-resolution[C]//In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City:IEEE,2018:2473-2481.
[16]ZHANG Y L,LI K P,LI K,et al.Image super-resolution using very deep residual channel attention networks[C]//Proceedings of the European Conference on Computer Vision(ECCV).Munich:Springer,2018:286-301.
[17]DAI T,CAI J R,ZHANG Y B,et al.Second-order attention network for single image super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Long Beach:IEEE,2019:11065-11074.
[18]NIU B,WEN W L,REN W Q,et al.Single image super-resolution via a holistic attention network[C]//Proceedings of the European Conference on Computer Vision(ECCV).Glasgow:Springer,2020:191-207.
[19]LI Q,ZHAO F Y.Super-resolution of Remote Sensing Images Based on Multi-path and Mixed Attention[J].Journal of Chinese Computer Systems,2023,44(7):1508-1513.
[20]MEI Y Q,FAN Y C,ZHOU Y Q,et al.Image super-resolution with cross-scale non-local attention and exhaustive self-exemplars mining[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.IEEE,2020:5690-5699.
[21]WANG Y,LI Y S,WANG G,et al.Multi-scale attention network for single image super-resolution[EB/OL].(2022-9-29) [2023-07-23].https://arxiv.org/abs/2209.14145.
[22]RAO Y M,ZHAO W L,TANG Y S,et al.HorNet:Efficient High-Order Spatial Interactions with Recursive Gated Convolutions[EB/OL].(2022-10-11) [2023-07-23].https://arxiv.org/abs/220 7.14284.
[23]LIU Z,MAO H Z,WU C Y,et al.A convnet for the 2020s[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.New Orleans:IEEE,2022:11976-11986.
[24]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[C]//Proceedings of the 31st International Confe-rence on Neural Information Processing Systems.Long Beach:MIT Press,2017:6000-6010.
[25]DOSOVITSKIY A,BEYER L,KOLESNIKOV A,et al.Animage is worth 16×16 words:Transformers for image recognition at scale [EB/OL].(2021-06-03) [2023-07-23].https://arxiv.org/abs/2010.11929.
[26]RADFORD A,KIM J W,HALLACY C,et al.Learning Transferable Visual Models From Natural Language Supervision[EB/OL].(2021-02-26) [2023-07-23].https://arxiv.org/abs/2103.00020v1.
[27]LI J S,XIA X,LI W,et al.Nextvit:Next generation visiontransformer for efficient deployment in realistic industrial scenarios[EB/OL].(2022-08-16) [2023-07-23].https://arxiv.org/abs/2207.05501.
[28]GUO J Y,HAN K,WU H,et al.Cmt:Convolutional neural networks meet vision transformers[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.New Orleans:IEEE,2022:12175-12185.
[29]LIU Z,LIN Y T,CAO Y,et al.Swin transformer:Hierarchical vision transformer using shifted windows[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.IEEE,2021:10012-10022.
[30]RAO Y M,ZHAO W L,ZHU Z,et al.Global filter networks for image classification[C]//Proceedings of the 33rd International Conference on Neural Information Processing Systems.MIT Press,2021:980-993.
[31]SHI W Z,CABALLERO J,HUSZÁR F,et al.Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE,2016:1874-1883.
[32]ZHOU S C,ZHANG J W,ZUO W M,et al.Cross-scale internal graph neural network for image super-resolution[C]//Procee-dings of the Thirty-fourth International Conference on Neural Information Processing Systems.MIT Press,2020:3499-3509.
[33]AGUSTSSON E,TIMOFTE R.Ntire 2017 challenge on single image super-resolution:Dataset and study[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.Honolulu:IEEE,2017:126-135.
[34]BEVILACQUA M,ROUMY A,GUILLEMOT C,et al.Low-complexity singleimage super-resolution based on nonnegative neighbor embedding[C]//British Machine Vision Conference.Surrey:British Machine Vision Association,2012:135.1-135.10.
[35]ZEYDE R,ELAD M,PROTTER M.On single image scale-up using sparse-representations[C]//Curves and Surfaces:7th International Conference.Avignon:Springer,2010:711-730.
[36]MARTIN D,FOWLKES C,TAL D,et al.A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//Proceedings Eighth IEEE International Conference on Computer Vision.Vancouver:IEEE,2001:416-423.
[37]HUANG J B,SINGH A,AHUJA N.Single image super-resolution from transformed self-exemplars[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Boston:IEEE,2015:5197-5206.
[38]MATSUI Y,ITO K,ARAMAKI Y,et al.Sketch-based manga retrieval using manga109 dataset[J].Multimedia Tools and Applications,2017,76:21811-21838.
[39]YANG Y,NEWSAM S.Bag-of-visual-words and spatial extensions for land-use classification[C]//Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems.New York:ACM,2010:270-279.
[40]ZOU Q,NI L Y,ZHANG T,et al.Deep learning based feature selection for remote sensing scene classification[J].IEEE Geoscience and Remote Sensing Letters,2015,12(11):2321-2325.
[41]XIA G S,HU J W,HU F,et al.Aid:A benchmark data set for performance evaluation of aerial scene classification[J].IEEE Transactions on Geoscience and Remote Sensing,2017,55(7):3965-3981.
[42]DAI D X,YANG W.Satellite image classification via two-layer sparse coding with biased image representation[J].IEEE Geoscience and Remote Sensing Letters,2010,8(1):173-176.
[43]CHENG G,HAN J W,LU X Q,et al.Remote sensing image scene classification:Benchmark and state of the art[J].Procee-dings of the IEEE,2017,105(10):1865-1883.
[44]WANG Z,BOVIK A C,SHEIKH H R,et al.Image quality assessment:from error visibility to structural similarity[J].IEEE Transactions on Image Processing,2004,13(4):1-14.
[45]SHEIKH H R,BOVIK A C.Image information and visual quality[J].IEEE Transactions on Image Processing,2006,15(2):430-444.
[46]ZHANG R,ISOLA P,EFROS A A,et al.The unreasonable effectiveness of deep features as a perceptual metric[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City:IEEE,2018:586-595.
[47]MEI Y Q,FAN Y C,ZHOU Y Q.Image super-resolution with non-local sparse attention[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.IEEE,2021:3517-3526.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发