基于感知损失的遥感图像全色锐化反馈网络

doi:10.11896/jsjkx.200700112

摘要/Abstract

摘要： 全色锐化旨在通过一个高分辨率的单通道全色图像(Panchromatic,PAN)锐化一个低分辨率的多通道多光谱图像(Multispectral,MS),得到一个高分辨率的多通道多光谱图像(High Resolution Multispectral,HRMS),这是遥感图像处理中的重要任务。文中提出了一个基于感知损失的反馈网络,首先对PAN图像和MS图像分别提取细节信息和光谱信息,然后将其合并后利用堆叠的上下采样层和密集连接进行信息融合,利用反馈连接使高层次的信息丰富低层次的信息,最后重建HRMS图像。与传统全色锐化算法相比,所提算法将PAN图像和HRMS图像一起作为网络输出的监督,通过求取PAN图像和网络重建HRMS图像的感知损失使输出图像含有更丰富的空间细节信息。无论是在客观指标还是视觉感受方面,与现有广泛使用的算法相比,所提算法都有更好的效果。

关键词: 反馈, 感知损失, 卷积神经网络, 全色锐化

Abstract: Pansharpening aims to sharpen a low-resolution multi-channel multispectral(MS) image from a high-resolution single-channel panchromatic(PAN) image to obtain a high resolution multispectral(HRMS) image,which is an important task in remote sensing image processing.A feedback network based on perceptual loss is proposed.First,the detail information and the spectral information are extracted from the PAN image and the MS image respectively,and then they are combined to use the stacked up-and down-sampling layers and dense connections for information fusion.The feedback connection is used to enrich the low-level information with high-level information.Finally,the HRMS image is reconstructed.Compared with the traditional pansharpening algorithms,the proposed algorithm uses the PAN image and the HRMS image as the supervision of the network,and the output image contains richer spatial detail information by obtaining the perceptual loss of the PAN image and the network reconstructed HRMS image.The experimental results show that the proposed algorithm has better results than the widely used algorithms both in objective evaluation and visual perception.

Key words: Convolutional neural network, Feedback, Pansharpening, Perceptual loss

中图分类号:

TP751.1

王乐, 杨晓敏. 基于感知损失的遥感图像全色锐化反馈网络[J]. 计算机科学, 2021, 48(8): 91-98. https://doi.org/10.11896/jsjkx.200700112

WANG Le, YANG Xiao-min. Remote Sensing Image Pansharpening Feedback Network Based on Perceptual Loss[J]. Computer Science, 2021, 48(8): 91-98. https://doi.org/10.11896/jsjkx.200700112

参考文献

[1]CHEN J Y,TIAN J Q.Vegetation Classification Based on High-resolution Satellite Image [J].Journal of Remote Sensing,2007,11(2):221-227.
[2]WANG X Y,LIU Y,JIANG Z Y.An IHS Fusion Method based on Structural Similarity[J].Remote Sensing Technology and Application,2011,26(5):670-676.
[3]ZENG Y Y,HE J N.Remote Sensing Image Fusion MethodBased on Regional Wavelet Statistical Features[J].Computer Engineering,2011,37(19):198-200.
[4]SUN P,DENG L,NIE J.Multi-scale Remote Sensing Image Fusion Method based on Region Segmentation[J].Remote Sensing Technology and Application,2012,27(6):844-849.
[5]LI S T,YANG B.A new pan-sharpening method using a compressed sensing technique[J].IEEE Transactions on Geoscience and Remote Sensing,2011,49(2):738-746.
[6]YIN H T.Sparse representation based pan-sharpening with details injection model[J].Signal Processing,2015,113:218-227.
[7]MASI G,COZZOLINO D,VERDOLIVA L,,et al.Pansharpening by Convolutional Neural Networks[J].Remote Sensing,2016,8(7):594-616.
[8]ZHANG Y,LIU C,SUN M,et al.Pan-Sharpening Using an Efficient Bidirectional Pyramid Network[J].IEEE Transactions on Geoscience and Remote Sensing,2019,57(8):5549-5563.
[9]LIU X,LIU Q,WANG Y.Remote Sensing Image Fusion Based on Two-stream Fusion Network[J].Lecture Notes in Computer Science,2018,10704:428-439.
[10]GUO Y,YE F,GONG H.Learning an Efficient ConvolutionNeural Network for Pan-sharpening[J].Algorithms,2019,12(1):16.
[11]YAO W,ZENG Z G,LIAN C,et al.Pixel-wise regression using U-Net and its application on pan-sharpening[J].Neurocompu-ting,2018,312:364-371.
[12]HE L,RAO Y,LI J,et al.Pansharpening via Detail Injection Based Convolutional Neural Net-works[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sen-sing,2019,12(4):1188-1204.
[13]CARREIRA J,AGRAWAL P,FRAGKIADAKI K,et al.Human Pose Estimation with Iterative Error Feedback[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE Press,2016:4733-4742.
[14]HARIS M,SHAKHNAROVICH G,UKITA N.Deep back-projection networks for super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Utah:IEEE Press,2018:1664-1673.
[15]HAN W,CHANG S,LIU D,et al.Image Super-Resolution via Dual-State Recurrent Networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Utah:IEEE Press,2018:1654-1663.
[16]LI Z,YANG J,LIU Z,et al.Feedback Network for Image Super-Resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.California:IEEE Press,2019:3867-3876.
[17]JOHNSON J,ALAHI A,LI F F.Perceptual Losses for Real-Time Style Transfer and Super-Resolution[J].Lecture Notes in Computer Science,2016,9906:694-711.
[18]WALD L,RANCHIN T,MANGOLINI M.Fusion of satelliteimages of different spatial resolutions:Assessing the quality of resulting images[J].Photogrammetric Engineering and Remote Sensing,1997,63(6):691-699.
[19]GATYS L A,ECKER A S,BETHGE M.Image Style Transfer Using Convolutional Neural Networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE Press,2016:2414-2423.
[20]KANG X,LI S,BENEDIKTSSON J A.Pansharpening withMatting Model[J].IEEE Transactions on Geoence and Remote Sensing,2014,52(8):5088-5099.
[21]JIN B,KIM G,CHO N I.Wavelet-domain satellite image fusion based on a generalized fusion equation[J].Journal of Applied Remote Sensing,2014,8(1):080599.
[22]TU T M,HUANG P S,HUNG C L,et al.A Fast Intensity-Hue-Saturation Fusion Technique With Spectral Adjustment for IKONOS Imagery[J].IEEE Geoence and Remote Sensing Letters,2004,1(4):309-312.
[23]JR P S C,KWARTENG A Y.Extracting spectral contrast in Landsat Thematic Mapper image data using selective principal component analysis[J].Photogrammetric Engineering and Remote Sensing,1989,55(3):339-348.
[24]SHAH V P,YOUNAN N H,KING R L.An Efficient Pan-Sharpening Method via a Combined Adaptive PCA Approach and Contourlets[J].IEEE Transactions on Geoence & Remote Sensing,2008,46(5):1323-1335.
[25]WEI Y,YUAN Q,SHEN H,et al.Boosting the Accuracy of Multispectral Image Pansharpening by Learning a Deep Residual Network[J].IEEE Geoence and Remote Sensing Letters,2017,14(10):1795-1799.

相关文章 15

[1]	周乐员, 张剑华, 袁甜甜, 陈胜勇. 多层注意力机制融合的序列到序列中国连续手语识别和翻译 Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion 计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026
[2]	李宗民, 张玉鹏, 刘玉杰, 李华. 基于可变形图卷积的点云表征学习 Deformable Graph Convolutional Networks Based Point Cloud Representation Learning 计算机科学, 2022, 49(8): 273-278. https://doi.org/10.11896/jsjkx.210900023
[3]	陈泳全, 姜瑛. 基于卷积神经网络的APP用户行为分析方法 Analysis Method of APP User Behavior Based on Convolutional Neural Network 计算机科学, 2022, 49(8): 78-85. https://doi.org/10.11896/jsjkx.210700121
[4]	朱承璋, 黄嘉儿, 肖亚龙, 王晗, 邹北骥. 基于注意力机制的医学影像深度哈希检索算法 Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism 计算机科学, 2022, 49(8): 113-119. https://doi.org/10.11896/jsjkx.210700153
[5]	檀莹莹, 王俊丽, 张超波. 基于图卷积神经网络的文本分类方法研究综述 Review of Text Classification Methods Based on Graph Convolutional Network 计算机科学, 2022, 49(8): 205-216. https://doi.org/10.11896/jsjkx.210800064
[6]	陈俊, 何庆, 李守玉. 基于自适应反馈调节因子的阿基米德优化算法 Archimedes Optimization Algorithm Based on Adaptive Feedback Adjustment Factor 计算机科学, 2022, 49(8): 237-246. https://doi.org/10.11896/jsjkx.210700150
[7]	张颖涛, 张杰, 张睿, 张文强. 全局信息引导的真实图像风格迁移 Photorealistic Style Transfer Guided by Global Information 计算机科学, 2022, 49(7): 100-105. https://doi.org/10.11896/jsjkx.210600036
[8]	戴朝霞, 李锦欣, 张向东, 徐旭, 梅林, 张亮. 基于DNGAN的磁共振图像超分辨率重建算法 Super-resolution Reconstruction of MRI Based on DNGAN 计算机科学, 2022, 49(7): 113-119. https://doi.org/10.11896/jsjkx.210600105
[9]	刘月红, 牛少华, 神显豪. 基于卷积神经网络的虚拟现实视频帧内预测编码 Virtual Reality Video Intraframe Prediction Coding Based on Convolutional Neural Network 计算机科学, 2022, 49(7): 127-131. https://doi.org/10.11896/jsjkx.211100179
[10]	徐鸣珂, 张帆. Head Fusion:一种提高语音情绪识别的准确性和鲁棒性的方法 Head Fusion:A Method to Improve Accuracy and Robustness of Speech Emotion Recognition 计算机科学, 2022, 49(7): 132-141. https://doi.org/10.11896/jsjkx.210100085
[11]	金方焱, 王秀利. 融合RACNN和BiLSTM的金融领域事件隐式因果关系抽取 Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM 计算机科学, 2022, 49(7): 179-186. https://doi.org/10.11896/jsjkx.210500190
[12]	杨玥, 冯涛, 梁虹, 杨扬. 融合交叉注意力机制的图像任意风格迁移 Image Arbitrary Style Transfer via Criss-cross Attention 计算机科学, 2022, 49(6A): 345-352. https://doi.org/10.11896/jsjkx.210700236
[13]	杨健楠, 张帆. 一种结合双注意力机制和层次网络结构的细碎农作物分类方法 Classification Method for Small Crops Combining Dual Attention Mechanisms and Hierarchical Network Structure 计算机科学, 2022, 49(6A): 353-357. https://doi.org/10.11896/jsjkx.210200169
[14]	卿朝进, 杜艳红, 叶青, 杨娜, 张岷涛. 存在CSI估计错误的增强型ELM叠加CSI反馈方法 Enhanced ELM-based Superimposed CSI Feedback Method with CSI Estimation Errors 计算机科学, 2022, 49(6A): 632-638. https://doi.org/10.11896/jsjkx.210800036
[15]	王杉, 徐楚怡, 师春香, 张瑛. 基于CNN-LSTM的卫星云图云分类方法研究 Study on Cloud Classification Method of Satellite Cloud Images Based on CNN-LSTM 计算机科学, 2022, 49(6A): 675-679. https://doi.org/10.11896/jsjkx.210300177

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed