基于多尺度双重自注意力的遥感影像变化检测

doi:10.11896/jsjkx.231000097

计算机科学 ›› 2024, Vol. 51 ›› Issue (11A): 231000097-9.doi: 10.11896/jsjkx.231000097

• 图像处理&多媒体技术 • 上一篇下一篇

基于多尺度双重自注意力的遥感影像变化检测

史经业¹, 左一平^2,3, 支瑞聪^2,3, 刘吉强¹, 张梦鸽⁴

1 北京交通大学智能交通数据安全与隐私保护技术北京市重点实验室北京 100044
2 北京科技大学计算机与通信工程学院北京 100083
3 材料领域知识工程北京市重点实验室(北京科技大学) 北京 100083
4 郑州大学计算机与人工智能学院郑州 450001

出版日期:2024-11-16 发布日期:2024-11-13
通讯作者: 刘吉强(jqliu2015@126.com)
作者简介:(shijy@geovis.com.cn)
基金资助:
国家科技支撑计划(2018YFC0823002);中央高校基本科研业务费专项资金(FRF-TP-20-10B,FRF-GF-19-010A)

Multi-scale Dual Self-attention Based Remote Sensing Image Change Detection

SHI Jingye¹, ZUO Yiping^2,3, ZHI Ruicong^2,3, LIU Jiqiang¹, ZHANG Mengge⁴

1 Beijing Key Laboratory of Intelligent Transportation Data Security and Privacy Protection Technology,Beijing Jiaotong University,Beijing 100044,China
2 School of Computer and Communication Engineering,University of Science and Technology Beijing,Beijing 100083,China
3 Beijing Key Laboratory of Knowledge Engineering for Materials Science(University of Science and Technology Beijing),Beijing 100083,China
4 School of Computer and Artificial Intelligence,Zhengzhou University,Zhengzhou 450001,China

Online:2024-11-16 Published:2024-11-13
About author:SHI Jingye,born in 1982,Ph.D candidate.His main research interests include big data management of aerospace information,intelligent analysis of remote sensing images.
LIU Jiqiang,born in 1973,Ph.D,professor,Ph.D supervisor.His main research interests include trusted computing, privacy protection,cloud computing security and intelligent analysis.
Supported by:
National Key Technology Research and Development Program of the Ministry of Science and Technology of China(2018YFC0823002) and Fundamental Research Funds for the Central Universities of Ministry of Education of China(FRF-TP-20-10B,FRF-GF-19-010A).

摘要/Abstract

摘要： 针对遥感影像地物目标尺度不一、上下文信息不足和边缘细节信息难以恢复等问题,提出一种基于多尺度双重自注意力的像素级变化检测网络(Pixel-based change detection Network,PixelNet)实现遥感影像变化检测任务。一方面,使用基于混合空洞卷积的多尺度特征金字塔提取卷积特征,并加入双重自注意力模块获取通道和空间注意力,兼顾细节和语义信息的同时增加特征感受野,进一步增加了全局上下文信息。另一方面,为了优化地物目标的边界圆滑模糊问题,通过边缘感知损失与加权对比损失的自动化联合训练,实现新的边缘修复模块。针对样本不均衡问题提出了带阈值的加权均衡采样的数据处理策略,以减轻变化像素数目远远小于未变化像素数目造成的网络训练倾斜问题。在遥感影像数据集CDD和LEVIR-CD上通过实验证明,所提像素级变化检测网络PixelNet在遥感变化检测任务上的主观视觉效果及客观评价指标优于SOTA的检测结果。在CDD数据集上检测精度达到98.0%,F1分数达到96.7%;在LEVIR-CD数据集上检测精度达到95.8%,F1分数为87.2%。该网络有效解决了遥感变化检测中样本不平衡、双时相特征上下文信息不足、边缘难例分类错误等问题。

关键词: 变化检测, 遥感影像, 混合空洞卷积, 双重自注意力, 边缘修复

Abstract: Aiming at the problems of different scales of ground objects,insufficient context information and difficult recovery of edge details in remote sensing images,a pixel-level change detection network(PixelNet) based on multi-scale dual self-attention is proposed to realize the task of remote sensing images change detection.On the one hand,the multi-scale feature pyramid based on hybrid cavity convolution is used to extract the convolution features,and the dual self-attention module is added to obtain the channel and spatial attention.The feature receptive field is increased while considering the details and semantic information,and the global context information is further increased.On the other hand,in order to optimize the boundary smoothing fuzzy problem of ground object,a new edge repair module is implemented through automatic joint training of edge sensing loss and weighted contrast loss.To solve the problem of sample imbalance,a data processing strategy of weighted balanced sampling with threshold is proposed to reduce the skew problem of network training caused by the number of changed pixels is much smaller than that of unchanged pixels.Experiments on remote sensing image datasets CDD and LEVIR-CD show that the proposed PixelNet network outperforms SOTA in terms of subjective visual effects and objective evaluation indexes on remote sensing change detection tasks.The detection accuracy is 98.0% and F1 score is 96.7% on the CDD dataset,the accuracy reaches 95.8% and F1 score reaches 87.2% on the LEVIR-CD dataset.It effectively solves the problems of sample imbalance,lack of context information of biphasic features,and classification error of difficult edge examples in remote sensing change detection.

Key words: Change detection, Remote sensing images, Hybrid cavity convolution, Double self-attention, Edge repair

中图分类号:

TP753

史经业, 左一平, 支瑞聪, 刘吉强, 张梦鸽. 基于多尺度双重自注意力的遥感影像变化检测[J]. 计算机科学, 2024, 51(11A): 231000097-9. https://doi.org/10.11896/jsjkx.231000097

SHI Jingye, ZUO Yiping, ZHI Ruicong, LIU Jiqiang, ZHANG Mengge. Multi-scale Dual Self-attention Based Remote Sensing Image Change Detection[J]. Computer Science, 2024, 51(11A): 231000097-9. https://doi.org/10.11896/jsjkx.231000097

参考文献

[1]VAN DE VOORDE T,CANTERS F,POELMANS L,et al.Incorporating land-use mapping uncertainty in remote sensing based calibration of land-use change models[J].The International Archives of the Photogrammetry,Remote Sensing and Spatial Information Sciences,2013,2(1):7-12.
[2]FENG W Q,ZHANG Y J.Object-oriented Change Detection for Remote Sensing Images Based on Multi-scale Fusion[J].Acta Geodaetica et Cartographica Sinica,2015,44(10):1142-1151.
[3]WU J,LI B,NI W,et al.An adaptively weighted multi-feature method for object-based change detection in high spatial resolution remote sensing images[J].Remote Sensing Letters,2020,11(4):333-342.
[4]BAN Y,YOUSIF O.Change detection techniques:A review[J].Multitemporal Remote Sensing,2016,20:19-43.
[5]BRAMICH D M.A new algorithm for difference image analysis[J].Monthly Notices of the Royal Astronomical Society:Letters,2008,386(1):L77-L81.
[6]MA J,GONG M,ZHOU Z.Wavelet Fusion on Ratio Images forChange Detection in SAR Images[J].IEEE Geoscience & Remote Sensing Letters,2012,9(6):1122-1126.
[7]TEWKESBURY A P,COMBER A J,TATE N J,et al.A critical synthesis of remotely sensed optical image change detection techniques[J].Remote Sensing of Environment,2015,160:1-14.
[8]HUANG W,HUANG J L,WANG L H,et al.Remote sensingimage change detection based on change vector analysis of PCA component[J].Remote Sensing for Land & Resources,2016,28(1):22-27.
[9]DHARANI M,SREENIVASULU G.Land use and land coverchange detection by using principal component analysis and morphological operations in remote sensing applications[J].International Journal of Computers and Applications,2021,43(5):462-471.
[10]ZHU S,XIA X,ZHANG Q,et al.An image segmentation algorithm in image processing based on threshold segmentation[C]//2007 3th IEEE Conference on Signal-image Technologies and Internet-Based System.Shanghai,China,IEEE,2007:673-678.
[11]PENG D,ZHANG Y.Object-based change detection from satellite imagery by segmentation optimization and multi-features fusion[J].International Journal of Remote Sensing,2017,38(13):3886-3905.
[12]ZHANG Y,PENG D,HUANG X.Object-based change detection for VHR images based on multiscale uncertainty analysis[J].IEEE Geoscience and Remote Sensing Letters,2017,15(1):13-17.
[13]SEYDI S T,HASANLOU M,AMANI M.A new end-to-endmulti-dimensional CNN framework for land cover/land use change detection in multi-source remote sensing datasets[J].Remote Sensing,2020,12(12):2010.
[14]LI X,CHEN H,QI X,et al.H-DenseUNet:hybrid densely connected UNet for liver and tumor segmentation from CT volumes[J].IEEE transactions on medical imaging,2018,37(12):2663-2674.
[15]CHEN L C,PAPANDREOU G,KOKKINOS I,et al.Deeplab:Semantic image segmentation with deep convolutional nets,atrous convolution,and fully connected crfs[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,40(4):834-848.
[16]PENG D,ZHANG Y,GUAN H.End-to-end change detectionfor high resolution satellite images using improved UNet++[J].Remote Sensing,2019,11(11):1382.
[17]ZHOU Z,SIDDIQUEE M M R,TAJBAKHSH N,et al.Unet++:Redesigning skip connections to exploit multiscale features in image segmentation[J].IEEE Transactions on Medical Imaging,2019,39(6):1856-1867.
[18]FANG B,PAN L,KOU R.Dual Learning-Based SiameseFramework for Change Detection Using Bi-Temporal VHR Optical Remote Sensing Images[J].Remote Sensing,2019,11(11):1292.
[19]CHEN Y,OUYANG X,AGAM G.Mfcnet:End-to-end ap-proach for change detection in images[C]//2018 25th IEEE International Conference on Image Processing.Athens,Greece,IEEE,2018:4008-4012.
[20]HAN X,LEUNG T,JIA Y,et al.MatchNet:Unifying featureand metric learning for patch-based matching[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition.Boston,MA,USA,IEEE 2015:3279-3286.
[21]LONG J,SHELHAMER E,DARRELL T.Fully convolutional networks for semantic segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,39(4):640-651.
[22]CHEN L C,PAPANDREOU G,SCHROFF F,et al.Rethinking atrous convolution for semantic image segmentation[J].arXiv:1706.05587,2017.
[23]SZEGEDY C,LIU W,JIA Y,et al.Going deeper with convolutions[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition.Boston,MA,USA,IEEE 2015:1-9.
[24]CHEN J,YUAN Z,PENG J,et al.DASNet:Dual attentive fully convolutional Siamese networks for change detection in high-resolution satellite images[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2020,14:1194-1206.
[25]CSURKA G,LARLUS D,PERRONNIN F,et al.What is a good evaluation measure for semantic segmentation[C]//2013 Electronic Proceedings of the British Machine Vision Conference.London,British,BMVC,2013.
[26]BOKHOVKIN A,BURNAEV E.Boundary loss for remotesensing imagery semantic segmentation[C]//2019 In International Symposium on Neural Networks.Cham,Germany,Springer,2019:388-401.
[27]LEBEDEV M A,VIZILTER Y V,VYGOLOV O V,et al.Change detection in remote sensing images using conditional adversarial networks[J].International Archives of the Photogrammetry,Remote Sensing & Spatial Information Sciences,2018,42(2):565-571.
[28]CHEN H,SHI Z.A spatial-temporal attention-based method and a new dataset for remote sensing image change detection[J].Remote Sensing,2020,12(10):1662.
[29]DAUDT R C,LE SAUX B,BOULCH A.Fully convolutional siamese networks for change detection[C]//2018 25th IEEE International Conference on Image Processing.Athens,Greece,IEEE 2018:4063-4067.
[30]FANGS,LI K,SHAO J,et al.SNUNet-CD:A densely connected Siamese network for change detection of VHR images[J].IEEE Geoscience and Remote Sensing Letters,2021,19:1-5.
[31]BANDARA W G C,PATEL V M.A transformer-based siamese network for change detection[C]//2022 IEEE International Geoscience and Remote Sensing Symposium(IGARSS 2022).IEEE,2022:207-210.
[32]SHI Q,LIU M,LI S,et al.A deeply supervised attention metric-based network and an open aerial image dataset for remote sensing change detection[J].IEEE Transactions on Geoscience and Remote Sensing,2022,60:1-16.
[33]HAN C,WU C,GUO H,et al.HANet:A hierarchical attention network for change detection with bi-temporal very-high-resolution remote sensing images[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2023,16:3867-3878.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

基于多尺度双重自注意力的遥感影像变化检测

Multi-scale Dual Self-attention Based Remote Sensing Image Change Detection

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 0

Metrics

本文评价

推荐阅读 0