Computer Science ›› 2024, Vol. 51 ›› Issue (6A): 230400110-6.doi: 10.11896/jsjkx.230400110

• Image Processing & Multimedia Technolog • Previous Articles     Next Articles

Remote Sensing Image Fusion Combining Multi-scale Convolution Blocks and Dense Convolution Blocks

HOU Linhao, LIU Fan   

  1. College of Data Science,Taiyuan University of Technology,Jinzhong,Shanxi 030600,China
  • Published:2024-06-06
  • About author:HOU Linhao,born in 1996,postgra-duate.His main research interests include remote sensing image fusion and deep learning.
    LIU Fan,born in 1982,Ph.D,professor,is a member of CCF(No.49460M).Her main research interests include machine learning and remote sensing image fusion.

Abstract: The aim of remote sensing image fusion is to obtain high-resolution multispectral images with the same spectral resolution as multispectral images and the same spatial resolution as panchromatic images.Although deep learning has achieved remarkable results in remote sensing image fusion,the network cannot fully extract the rich spatial information in the image due to the limitation of the deep model network,which leads to the lack of spatial information in the fused image and low quality of the fusion result.Therefore,this paper introduces multi-scale blocks,where image features at different scales can be learned by convolutional kernels of different sizes,thus increasing the richness of the extracted features.Dense convolutional blocks are then introduced to achieve feature reuse through dense connections,reducing the loss of shallow feature information when the network is deep.In the feature fusion stage,the proposed method uses feature maps from different levels of the network as input to the feature fusion layer to improve the quality of the fused images.Comparison experiments are performed with six fusion algorithms on GE1 and QB datasets,and the experimental results show that the fused images of the proposed method retain spatial and spectral information better,and outperform the comparison methods in both subjective and objective evaluations.

Key words: Remote sensing image fusion, Deep learning, Multispectral images, Multiscale convolution block, Dense connection

CLC Number: 

  • TP391
[1]VIVONE G,DALLA MURA M,GARZELLI A,et al.A new benchmark based on recent advances in multispectral panshar-pening:Revisiting pansharpening with classical and emerging pansharpening methods[J].IEEE Geoscience and Remote Sensing Magazine,2020,9(1):53-81.
[2]LEUNG Y,LIU J,ZHANG J.An Improved Adaptive Intensity-Hue-Saturation Method for the Fusion of Remote Sensing Images[J].IEEE Geoscience and Remote Sensing Letters,2014,11(5):985-989.
[3]SHAH V P,YOUNAN N H,KING R L.An Efficient Pan-Sharpening Method via a Combined Adaptive PCA Approach and Contourlets[J].IEEE Transactions on Geoscience and Remote Sensing,2008,46(5):1323-1335.
[4]LABEN C A,BROWER B V.Process for Enhancing the Spatial Resolution of Multispectral Imagery Using Pan-Sharpening:US Patent 6,011,875,4[P].2000-01.
[5]CHOI J,YU K,KIM Y.A new adaptive component-sub-stitution-based satellite image fusion by using partial replacement[J].IEEE Transaction on Geoscience and Remote Sensing,2011,49(1):295-309.
[6]KIM Y,LEE C,HAN D,et al.Improved Additive-WaveletImage Fusion[C]//IEEE Geoscience and Remote Sensing Letters.2011:263-267.
[7]LI S T,KWOK JAMES T,WANG Y.Using the discrete wavelet frame transform to merge Landsat TM and SPOT panchromatic images[J].Information Fusion,2002,3(1):17-23.
[8]ZHANG J,CHEN H T,LIU F.Remote Sensing Image Fusion Based on Multivariate Empirical Mode Decomposition and Weighted Least Squares Filter[J].Acta Photonica Sinica,2019,48(5):510003.
[9]MASI G,COZZOLINO D,VERDOLIVA L,et al.Pansharpening by convolutional neural networks[J].Remote Sensing,2016,8(7):594.
[10]YUAN Q,WEI Y,MENG X,et al.A Multiscale and Multidepth Convolutional Neural Network for Remote Sensing Imagery Pan-Sharpening[C]//IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.2018:978-989.
[11]SHAO Z,CAI J.Remote Sensing Image Fusion With DeepConvolutional Neural Network[C]//IEEE Journal of Selected Topi-cs in Applied Earth Observations and Remote Sensing.2018:1656-1669.
[12]LI M,LIU F,LI J Z.Combining Convolutional Attention Module and Convolutional Auto-encoder for Detail Injection Remote Sensing Image Fusion[J].Acta Photonica Sinica,2022,51(6):0610005.
[13]LI J,FANG F,LI J,et al.MDCN:Multi-Scale Dense Cross Network for Image Super-Resolution[J].IEEE Transactions on Circuits and Systems for Video Technology,2021,31(7):2547-2561.
[14]CHANG C Y,CHIEN S Y.Multi-scale Dense Network for Single-image Super-resolution[C]//2019 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP 2019).IEEE,2019.
[15]WALD L,RANCHIN T,MANGOLINI M.Fusion of satelliteimages of different spatial resolutions:Assessing the quality of resulting images[J].Photogrammetric Engineering and Remote Sensing,1997,63:691-699.
[16]GARZELLI A,ENCINI F,CAPOBIANCO L.Optimal MMSE Pan Sharpening of Very High Resolution Multispectral Images[C]//IEEE Transactions on Geoscience and Remote Sensing.2008:228-236.
[17]AIAZZI B,BARONTI S,SELVA M.Improving component sub-stitution pansharpening through multivariate regression of MS+Pan data[J].IEEE Trans.Geosci.Remote Sens.,2007,45(10):3230-3239.
[18]VIVONE G,RESTAINO R,DALLA MURA M,et al.Contrast and error-based fusion schemes for multispectral image pansharpening[J].IEEE Geoscience and Remote Sensing Letters,2013,11(5):930-934.-
[19]JIN Z,ZHUO Y,ZHANG T,et al.Remote Sensing Pansharpening by Full-Depth Feature Fusion[J].Remote Sensing,2022,14(3).
[1] HUANG Haixin, CAI Mingqi, WANG Yuyao. Review of Point Cloud Semantic Segmentation Based on Graph Convolutional Neural Networks [J]. Computer Science, 2024, 51(6A): 230400196-7.
[2] WANG Yingjie, ZHANG Chengye, BAI Fengbo, WANG Zumin. Named Entity Recognition Approach of Judicial Documents Based on Transformer [J]. Computer Science, 2024, 51(6A): 230500164-9.
[3] LIANG Fang, XU Xuyao, ZHAO Kailong, ZHAO Xuanfeng, ZHANG Guijun. Remote Template Detection Algorithm and Its Application in Protein Structure Prediction [J]. Computer Science, 2024, 51(6A): 230600225-7.
[4] PENG Bo, LI Yaodong, GONG Xianfu, LI Hao. Method for Entity Relation Extraction Based on Heterogeneous Graph Neural Networks and TextSemantic Enhancement [J]. Computer Science, 2024, 51(6A): 230700071-5.
[5] ZHANG Tianchi, LIU Yuxuan. Research Progress of Underwater Image Processing Based on Deep Learning [J]. Computer Science, 2024, 51(6A): 230400107-12.
[6] WANG Guogang, DONG Zhihao. Lightweight Image Semantic Segmentation Based on Attention Mechanism and Densely AdjacentPrediction [J]. Computer Science, 2024, 51(6A): 230300204-8.
[7] WANG Li, CHEN Gang, XIA Mingshan, HU Hao. DUWe:Dynamic Unknown Word Embedding Approach for Web Anomaly Detection [J]. Computer Science, 2024, 51(6A): 230300191-5.
[8] LYU Yiming, WANG Jiyang. Iron Ore Image Classification Method Based on Improved Efficientnetv2 [J]. Computer Science, 2024, 51(6A): 230600212-6.
[9] YANG Xiuzhang, WU Shuai, REN Tianshu, LIAO Wenjing, XIANG Meiyu, YU Xiaomin, LIU Jianyi, CHEN Dengjian. Complex Environment License Plate Recognition Algorithm Based on Improved Image Enhancement and CNN [J]. Computer Science, 2024, 51(6A): 220200162-7.
[10] SONG Zhen, WANG Jiqiang, HOU Moyu, ZHAO Lin. Conveyor Belt Defect Detection Network Combining Attention Mechanism with Line Laser Assistance [J]. Computer Science, 2024, 51(6A): 230800115-6.
[11] WU Chunming, LIU Yali. Method for Lung Nodule Detection on CT Images Using Improved YOLOv5 [J]. Computer Science, 2024, 51(6A): 230500019-6.
[12] YIN Xudong, CHEN Junyang, ZHOU Bo. Study on Industrial Defect Augmentation Data Filtering Based on OOD Scores [J]. Computer Science, 2024, 51(6A): 230700111-7.
[13] QIAO Hong, XING Hongjie. Attention-based Multi-scale Distillation Anomaly Detection [J]. Computer Science, 2024, 51(6A): 230300223-11.
[14] SI Jia, LIANG Jianfeng, XIE Shuo, DENG Yingjun. Research Progress of Anomaly Detection in IaaS Cloud Operation Driven by Deep Learning [J]. Computer Science, 2024, 51(6A): 230400016-8.
[15] DUAN Pengsong, DIAO Xianguang, ZHANG Dalong, CAO Yangjie, LIU Guangyi, KONG Jinsheng. WiCare:Non-contact Fall Monitoring Model for Elderly in Toilet [J]. Computer Science, 2024, 51(6A): 230700044-8.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!