计算机科学 ›› 2024, Vol. 51 ›› Issue (11A): 240200097-6.doi: 10.11896/jsjkx.240200097

• 图像处理&多媒体技术 • 上一篇    下一篇

基于双重注意力机制的遥感图像时空融合方法

范学晶, 薛笑荣, 杜意超   

  1. 辽宁工业大学电子与信息工程学院 辽宁 锦州 121001
  • 出版日期:2024-11-16 发布日期:2024-11-13
  • 通讯作者: 薛笑荣(xr_986@163.com )
  • 作者简介:(734057293@qq.com)
  • 基金资助:
    辽宁省科技计划项目(2021JH2/10200023);辽宁省教育厅科研重点项目(LJKZ0618)

Spatiotemporal Fusion Method for Remote Sensing Images Based on Dual Attention Mechanisms

FAN Xuejing, XUE Xiaorong, DU Yichao   

  1. School of Electronic and Information Engineering,Liaoning University of Technology,Jinzhou,Liaoning 121001,China
  • Online:2024-11-16 Published:2024-11-13
  • About author:FAN Xuejing,born in 1996,postgra-duate.Her main research interests include image processing and pattern recognition.
    XUE Xiaorong,born in 1974,Ph.D,professor,is a member of CCF(No.14043M).His main research interests include image processing and pattern recognition,etc.
  • Supported by:
    Science and Technology Plan Projectof Liaoning Province(2021JH2/10200023)and Key Project of Scientific Research of Education Department of Liaoning Province(LJKZ0618).

摘要: 由于遥感成像技术条件的限制,难以同时获得既有高时间分辨率又具有高空间分辨率序列的遥感影像,通过时空融合技术可以生成同时具有高时间和高空间分辨率的遥感图像。近年来,时空融合方法层出不穷,这些方法效果良好,但在特征提取有效信息方面仍有不足。针对此问题,提出了一种基于双重注意力机制改进的深度学习模型(ADCSTFN),使得模型在全局的保留和细节场景的重建能力都得到了提高。在实验中,采用Landsat和MODIS数据为研究对象,使用两个开源数据集和一个本地数据集对该方法进行测试,并与4种常用的时空融合方法进行比较。实验结果表明,文中提出的残差网络和双重注意力机制方法能更好地提取图像的有效信息,使用深监督损失函数缓解反向传播时梯度消失的问题,优化了学习过程,使得融合后的效果有了明显的提高。

关键词: 时空融合, 遥感图像, 双重注意力机制, Landsat, MODIS

Abstract: Due to the limitations of remote sensing imaging technology conditions,it is difficult to obtain sequences of remote sensing images with both high temporal and high spatial resolution simultaneously. However,spatiotemporal fusion techniques can generate remote sensing images with high temporal and high spatial resolution. In recent years,various spatiotemporal fusion methods have emerged,showing good performance but still falling short in effectively extracting meaningful information in feature extraction. A deep learning model based on dual attention mechanism improvement (ADCSTFN) is proposed to address this issue,which enhances the model's ability to preserve global information and reconstruct detailed scenes. In the experiments,Landsat and MODIS data were used as the research subjects,and the proposed method was tested using two open-source datasets and a local dataset,and compared with four commonly used spatiotemporal fusion methods. Experimental results show that the resi-dual network and dual attention mechanism proposed in this paper better extract effective information from images. The use of a deep supervision loss function mitigates the issue of vanishing gradients during backpropagation,optimizes the learning process,and significantly improves the fusion results.

Key words: Spatiotemporal fusion, Remote sensing images, Dual attention mechanism, Landsat, MODIS

中图分类号: 

  • TP391
[1]GONG P.Some Frontier Issues in Remote Sensing Science and Technology[J].Journal of Remote Sensing,2009,13(1):13-23.
[2]TONG Q,ZHANG B,ZHANG L.Frontier Advances in Hyperspectral Remote Sensing in China[J].Journal of Remote Sensing,2016,20(5):689-707.
[3]WANG S.Research on Spatiotemporal Fusion Algorithms Based on Multi-source High-Resolution Satellite Remote Sensing Images [D].Beihua Aerospace Industry College,2023.
[4]LI C,SONG H,ZHANG K,et al.Conditional Generative Adversarial Networks for Spatiotemporal Fusion of Remote Sensing Images[J].Journal of Image and Graphics,2021,26(3):714-726
[5]SUN Y.Research on Spatiotemporal Fusion Algorithms for Remote Sensing Data-A Case Study with Landsat and MODIS Data [C]//China University of Mining and Technology,2019.
[6]HUANG B,ZHAO Y.Current Status and Prospects of Spatio-temporal Fusion Research on Multi-source Satellite Remote Sensing Images[J].Journal of Geomatics,2017,46(10):1492-1499.
[7]ZHANG H K,HUANG B,ZHANGM,et al.A generalization of spatial andtemporal fusion methods for remotelysensed surface parameters[J].International Journal of Remote Sensing,2015,36(17):4411-4445.
[8]SHEVYRNOGOV A,TREFOIS P,VYSOTSKAYA G.Multi-satellite Data Merge to Combine NOAA AVHRR Efficiency with Landsat-6 MSS Spatial Resolutionto StudyVegetationDynamics[J].Advances in Space Research,2000,26(7):1131-1133.
[9]GU X H,HAN L J,WANG J A.Remote Sensing Estimation of Corn Area by Wavelet Fusion of Medium and Low Resolution Images[J].Transactions of the Chinese Society of Agricultural Engineering,2012,28(3):203-209.
[10]ZHU X L,HELMER E H,GAO F,et al.A flexible spatiotemporal meth-od for fusing satellite images with different resolutions[J].Remote Sensing of Environment,2016,172:165-177.
[11]GAO F,MASEK J,SCHWALLER M,et al.On the blending of the Landsat and MODIS surface reflectance:predicting daily Landsat surlace reflectance[C]//IEEE Transactions on Geoscience and Remote Sensing.2006:2207-2218.
[12]ZHU X,CHEN J,GAO F.An enhanced spatial and temporaladaptive reflectance fusion model for complex heterogene?ous regionsJ7.Remote Sensing of Environment,2010,114(11);2610-2623.
[13]BUSETTO L,MERONI M,COLOMBO R.Combining medium and coarsespatial resolution satellite data to improve the estimation of sub-pixel NDVI time series[J].Remote Sensing of Environment,2008,112(1):118-131.
[14]ZEYDE R,ELAD M,PROTTER M.On single image scale-up using sparse-representations[C]//Proceedings of the 7th International Conference on Curves and Surfaces.Avignon,France:Springer,2012:711-730.
[15]SONG H,HUANG B.Spatiotemporal Satellite Image FusionThrough One-Pair Image Learning[J].IEEE Transactions on Geoscience and Remote Sensing,2013,51(4):1883-1896.
[16]SONG H,LIU Q,WANG G.Spatiotemporal Satellite Image Fusion Using Deep Convolutional Neural Networks[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2018,11(3):821-829.
[17]HE K M,ZHANG X Y,REN S Q,et al.Deep Residual Learning for Image Recognition[C]// IEEE Conference on Computer Vision and Pattern Recognition(CVPR).2015:770-778.
[18]LI Y,LI J,HE L.Single-sample-based Convolutional NeuralNetwork for Spatiotemporal Fusion of Remote Sensing Images[J].Journal of Remote Sensing,2022,26(8):1614-1623.
[19]SUN Z,OUYANG X,LI H,et al.Spatiotemporal Data Fusion Model of NDVI Based on Deep Learning[J].Journal of Resources and Ecology,2024,15(1):214-226.
[20]TAN Z Y,YUE P,DI L P,et al.Deriving high spatiotemporal remote sensing images using deep convolutional network[J].Remote Sensing,2018,10(7):1066.
[21]FU J,LIU J,TIAN H,et al.Dual Attention Network for Scene Segmentation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2018:3141-3149.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!