Computer Science ›› 2024, Vol. 51 ›› Issue (11A): 240200097-6.doi: 10.11896/jsjkx.240200097

• Image Processing & Multimedia Technology • Previous Articles     Next Articles

Spatiotemporal Fusion Method for Remote Sensing Images Based on Dual Attention Mechanisms

FAN Xuejing, XUE Xiaorong, DU Yichao   

  1. School of Electronic and Information Engineering,Liaoning University of Technology,Jinzhou,Liaoning 121001,China
  • Online:2024-11-16 Published:2024-11-13
  • About author:FAN Xuejing,born in 1996,postgra-duate.Her main research interests include image processing and pattern recognition.
    XUE Xiaorong,born in 1974,Ph.D,professor,is a member of CCF(No.14043M).His main research interests include image processing and pattern recognition,etc.
  • Supported by:
    Science and Technology Plan Projectof Liaoning Province(2021JH2/10200023)and Key Project of Scientific Research of Education Department of Liaoning Province(LJKZ0618).

Abstract: Due to the limitations of remote sensing imaging technology conditions,it is difficult to obtain sequences of remote sensing images with both high temporal and high spatial resolution simultaneously. However,spatiotemporal fusion techniques can generate remote sensing images with high temporal and high spatial resolution. In recent years,various spatiotemporal fusion methods have emerged,showing good performance but still falling short in effectively extracting meaningful information in feature extraction. A deep learning model based on dual attention mechanism improvement (ADCSTFN) is proposed to address this issue,which enhances the model's ability to preserve global information and reconstruct detailed scenes. In the experiments,Landsat and MODIS data were used as the research subjects,and the proposed method was tested using two open-source datasets and a local dataset,and compared with four commonly used spatiotemporal fusion methods. Experimental results show that the resi-dual network and dual attention mechanism proposed in this paper better extract effective information from images. The use of a deep supervision loss function mitigates the issue of vanishing gradients during backpropagation,optimizes the learning process,and significantly improves the fusion results.

Key words: Spatiotemporal fusion, Remote sensing images, Dual attention mechanism, Landsat, MODIS

CLC Number: 

  • TP391
[1]GONG P.Some Frontier Issues in Remote Sensing Science and Technology[J].Journal of Remote Sensing,2009,13(1):13-23.
[2]TONG Q,ZHANG B,ZHANG L.Frontier Advances in Hyperspectral Remote Sensing in China[J].Journal of Remote Sensing,2016,20(5):689-707.
[3]WANG S.Research on Spatiotemporal Fusion Algorithms Based on Multi-source High-Resolution Satellite Remote Sensing Images [D].Beihua Aerospace Industry College,2023.
[4]LI C,SONG H,ZHANG K,et al.Conditional Generative Adversarial Networks for Spatiotemporal Fusion of Remote Sensing Images[J].Journal of Image and Graphics,2021,26(3):714-726
[5]SUN Y.Research on Spatiotemporal Fusion Algorithms for Remote Sensing Data-A Case Study with Landsat and MODIS Data [C]//China University of Mining and Technology,2019.
[6]HUANG B,ZHAO Y.Current Status and Prospects of Spatio-temporal Fusion Research on Multi-source Satellite Remote Sensing Images[J].Journal of Geomatics,2017,46(10):1492-1499.
[7]ZHANG H K,HUANG B,ZHANGM,et al.A generalization of spatial andtemporal fusion methods for remotelysensed surface parameters[J].International Journal of Remote Sensing,2015,36(17):4411-4445.
[8]SHEVYRNOGOV A,TREFOIS P,VYSOTSKAYA G.Multi-satellite Data Merge to Combine NOAA AVHRR Efficiency with Landsat-6 MSS Spatial Resolutionto StudyVegetationDynamics[J].Advances in Space Research,2000,26(7):1131-1133.
[9]GU X H,HAN L J,WANG J A.Remote Sensing Estimation of Corn Area by Wavelet Fusion of Medium and Low Resolution Images[J].Transactions of the Chinese Society of Agricultural Engineering,2012,28(3):203-209.
[10]ZHU X L,HELMER E H,GAO F,et al.A flexible spatiotemporal meth-od for fusing satellite images with different resolutions[J].Remote Sensing of Environment,2016,172:165-177.
[11]GAO F,MASEK J,SCHWALLER M,et al.On the blending of the Landsat and MODIS surface reflectance:predicting daily Landsat surlace reflectance[C]//IEEE Transactions on Geoscience and Remote Sensing.2006:2207-2218.
[12]ZHU X,CHEN J,GAO F.An enhanced spatial and temporaladaptive reflectance fusion model for complex heterogene?ous regionsJ7.Remote Sensing of Environment,2010,114(11);2610-2623.
[13]BUSETTO L,MERONI M,COLOMBO R.Combining medium and coarsespatial resolution satellite data to improve the estimation of sub-pixel NDVI time series[J].Remote Sensing of Environment,2008,112(1):118-131.
[14]ZEYDE R,ELAD M,PROTTER M.On single image scale-up using sparse-representations[C]//Proceedings of the 7th International Conference on Curves and Surfaces.Avignon,France:Springer,2012:711-730.
[15]SONG H,HUANG B.Spatiotemporal Satellite Image FusionThrough One-Pair Image Learning[J].IEEE Transactions on Geoscience and Remote Sensing,2013,51(4):1883-1896.
[16]SONG H,LIU Q,WANG G.Spatiotemporal Satellite Image Fusion Using Deep Convolutional Neural Networks[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2018,11(3):821-829.
[17]HE K M,ZHANG X Y,REN S Q,et al.Deep Residual Learning for Image Recognition[C]// IEEE Conference on Computer Vision and Pattern Recognition(CVPR).2015:770-778.
[18]LI Y,LI J,HE L.Single-sample-based Convolutional NeuralNetwork for Spatiotemporal Fusion of Remote Sensing Images[J].Journal of Remote Sensing,2022,26(8):1614-1623.
[19]SUN Z,OUYANG X,LI H,et al.Spatiotemporal Data Fusion Model of NDVI Based on Deep Learning[J].Journal of Resources and Ecology,2024,15(1):214-226.
[20]TAN Z Y,YUE P,DI L P,et al.Deriving high spatiotemporal remote sensing images using deep convolutional network[J].Remote Sensing,2018,10(7):1066.
[21]FU J,LIU J,TIAN H,et al.Dual Attention Network for Scene Segmentation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2018:3141-3149.
[1] HU Pengfei, WANG Youguo, ZHAI Qiqing, YAN Jun, BAI Quan. Night Vehicle Detection Algorithm Based on YOLOv5s and Bistable Stochastic Resonance [J]. Computer Science, 2024, 51(9): 173-181.
[2] WANG Qian, HE Lang, WANG Zhanqing, HUANG Kun. Road Extraction Algorithm for Remote Sensing Images Based on Improved DeepLabv3+ [J]. Computer Science, 2024, 51(8): 168-175.
[3] ZHANG Yang, XIA Ying. Object Detection Method with Multi-scale Feature Fusion for Remote Sensing Images [J]. Computer Science, 2024, 51(3): 165-173.
[4] LIU Changxin, WU Ning, HU Lirui, GAO Ba, GAO Xueshan. Recursive Gated Convolution Based Super-resolution Network for Remote Sensing Images [J]. Computer Science, 2024, 51(2): 205-216.
[5] DONG Yan, WEI Minghong, GAO Guangshuai, LIU Zhoufeng, LI Chunlei. Remote Sensing Orineted Object Detection Method Based on Dual-label Assignment [J]. Computer Science, 2024, 51(11A): 240100058-9.
[6] SHI Jingye, ZUO Yiping, ZHI Ruicong, LIU Jiqiang, ZHANG Mengge. Multi-scale Dual Self-attention Based Remote Sensing Image Change Detection [J]. Computer Science, 2024, 51(11A): 231000097-9.
[7] YAN Yan, SUI Yi, SI Jianwei. Remote Sensing Image Pan-sharpening Method Based on Generative Adversarial Network [J]. Computer Science, 2023, 50(8): 133-141.
[8] HUANG Yaqun, ZHENG Peiyu, JIANG Murong, YANG Lei, LUO Jun. Reconstruction of Solar Speckle Image Combined with Gated Fusion Network and Residual Fourier Transform [J]. Computer Science, 2023, 50(11A): 220800229-7.
[9] WANG Kun-shu, ZHANG Ze-hui, GAO Tie-gang. Reversible Hidden Algorithm for Remote Sensing Images Based on Hachimoji DNA and QR Decomposition [J]. Computer Science, 2022, 49(8): 127-135.
[10] HUANG Yang-lin, HU Kai, GUO Jian-qiang, Peng ChengKey Laboratory of Intelligent Computing & Information Processing of Ministry of Education, Xiangtan University, Xiangtan, Hunan 411105,China. Liver CT Images Segmentation Based on Multi-scale Feature Fusion and Dual AttentionMechanism [J]. Computer Science, 2022, 49(11A): 210800162-9.
[11] ZHANG Man, LI Jie, ZHU Xin-zhong, SHEN Ji, CHENG Hao-tian. Augmentation Technology of Remote Sensing Dataset Based on Improved DCGAN Algorithm [J]. Computer Science, 2021, 48(6A): 80-84.
[12] WANG Zhen-wu, SUN Jai-jun,  YU Zhong-yi and BU Yi-ya. Review of Remote Sensing Image Classification Based on Support Vector Machine [J]. Computer Science, 2016, 43(9): 11-17.
[13] . Fast and Automatic Registration Method for Large Multi-spectral Remote Sensing Images [J]. Computer Science, 2012, 39(2): 61-65.
[14] XU Li-yan,WANG Jing,QIU Jun,SUN Quan-sen,XIA De-shen. Multi-spectral Remote Sensing Image Registration Based on Feature Point [J]. Computer Science, 2011, 38(7): 280-282.
[15] . [J]. Computer Science, 2009, 36(4): 268-272.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!