Computer Science ›› 2024, Vol. 51 ›› Issue (11A): 231000097-9.doi: 10.11896/jsjkx.231000097

• Image Processing & Multimedia Technology • Previous Articles     Next Articles

Multi-scale Dual Self-attention Based Remote Sensing Image Change Detection

SHI Jingye1, ZUO Yiping2,3, ZHI Ruicong2,3, LIU Jiqiang1, ZHANG Mengge4   

  1. 1 Beijing Key Laboratory of Intelligent Transportation Data Security and Privacy Protection Technology,Beijing Jiaotong University,Beijing 100044,China
    2 School of Computer and Communication Engineering,University of Science and Technology Beijing,Beijing 100083,China
    3 Beijing Key Laboratory of Knowledge Engineering for Materials Science(University of Science and Technology Beijing),Beijing 100083,China
    4 School of Computer and Artificial Intelligence,Zhengzhou University,Zhengzhou 450001,China
  • Online:2024-11-16 Published:2024-11-13
  • About author:SHI Jingye,born in 1982,Ph.D candidate.His main research interests include big data management of aerospace information,intelligent analysis of remote sensing images.
    LIU Jiqiang,born in 1973,Ph.D,professor,Ph.D supervisor.His main research interests include trusted computing, privacy protection,cloud computing security and intelligent analysis.
  • Supported by:
    National Key Technology Research and Development Program of the Ministry of Science and Technology of China(2018YFC0823002) and Fundamental Research Funds for the Central Universities of Ministry of Education of China(FRF-TP-20-10B,FRF-GF-19-010A).

Abstract: Aiming at the problems of different scales of ground objects,insufficient context information and difficult recovery of edge details in remote sensing images,a pixel-level change detection network(PixelNet) based on multi-scale dual self-attention is proposed to realize the task of remote sensing images change detection.On the one hand,the multi-scale feature pyramid based on hybrid cavity convolution is used to extract the convolution features,and the dual self-attention module is added to obtain the channel and spatial attention.The feature receptive field is increased while considering the details and semantic information,and the global context information is further increased.On the other hand,in order to optimize the boundary smoothing fuzzy problem of ground object,a new edge repair module is implemented through automatic joint training of edge sensing loss and weighted contrast loss.To solve the problem of sample imbalance,a data processing strategy of weighted balanced sampling with threshold is proposed to reduce the skew problem of network training caused by the number of changed pixels is much smaller than that of unchanged pixels.Experiments on remote sensing image datasets CDD and LEVIR-CD show that the proposed PixelNet network outperforms SOTA in terms of subjective visual effects and objective evaluation indexes on remote sensing change detection tasks.The detection accuracy is 98.0% and F1 score is 96.7% on the CDD dataset,the accuracy reaches 95.8% and F1 score reaches 87.2% on the LEVIR-CD dataset.It effectively solves the problems of sample imbalance,lack of context information of biphasic features,and classification error of difficult edge examples in remote sensing change detection.

Key words: Change detection, Remote sensing images, Hybrid cavity convolution, Double self-attention, Edge repair

CLC Number: 

  • TP753
[1]VAN DE VOORDE T,CANTERS F,POELMANS L,et al.Incorporating land-use mapping uncertainty in remote sensing based calibration of land-use change models[J].The International Archives of the Photogrammetry,Remote Sensing and Spatial Information Sciences,2013,2(1):7-12.
[2]FENG W Q,ZHANG Y J.Object-oriented Change Detection for Remote Sensing Images Based on Multi-scale Fusion[J].Acta Geodaetica et Cartographica Sinica,2015,44(10):1142-1151.
[3]WU J,LI B,NI W,et al.An adaptively weighted multi-feature method for object-based change detection in high spatial resolution remote sensing images[J].Remote Sensing Letters,2020,11(4):333-342.
[4]BAN Y,YOUSIF O.Change detection techniques:A review[J].Multitemporal Remote Sensing,2016,20:19-43.
[5]BRAMICH D M.A new algorithm for difference image analysis[J].Monthly Notices of the Royal Astronomical Society:Letters,2008,386(1):L77-L81.
[6]MA J,GONG M,ZHOU Z.Wavelet Fusion on Ratio Images forChange Detection in SAR Images[J].IEEE Geoscience & Remote Sensing Letters,2012,9(6):1122-1126.
[7]TEWKESBURY A P,COMBER A J,TATE N J,et al.A critical synthesis of remotely sensed optical image change detection techniques[J].Remote Sensing of Environment,2015,160:1-14.
[8]HUANG W,HUANG J L,WANG L H,et al.Remote sensingimage change detection based on change vector analysis of PCA component[J].Remote Sensing for Land & Resources,2016,28(1):22-27.
[9]DHARANI M,SREENIVASULU G.Land use and land coverchange detection by using principal component analysis and morphological operations in remote sensing applications[J].International Journal of Computers and Applications,2021,43(5):462-471.
[10]ZHU S,XIA X,ZHANG Q,et al.An image segmentation algorithm in image processing based on threshold segmentation[C]//2007 3th IEEE Conference on Signal-image Technologies and Internet-Based System.Shanghai,China,IEEE,2007:673-678.
[11]PENG D,ZHANG Y.Object-based change detection from satellite imagery by segmentation optimization and multi-features fusion[J].International Journal of Remote Sensing,2017,38(13):3886-3905.
[12]ZHANG Y,PENG D,HUANG X.Object-based change detection for VHR images based on multiscale uncertainty analysis[J].IEEE Geoscience and Remote Sensing Letters,2017,15(1):13-17.
[13]SEYDI S T,HASANLOU M,AMANI M.A new end-to-endmulti-dimensional CNN framework for land cover/land use change detection in multi-source remote sensing datasets[J].Remote Sensing,2020,12(12):2010.
[14]LI X,CHEN H,QI X,et al.H-DenseUNet:hybrid densely connected UNet for liver and tumor segmentation from CT volumes[J].IEEE transactions on medical imaging,2018,37(12):2663-2674.
[15]CHEN L C,PAPANDREOU G,KOKKINOS I,et al.Deeplab:Semantic image segmentation with deep convolutional nets,atrous convolution,and fully connected crfs[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,40(4):834-848.
[16]PENG D,ZHANG Y,GUAN H.End-to-end change detectionfor high resolution satellite images using improved UNet++[J].Remote Sensing,2019,11(11):1382.
[17]ZHOU Z,SIDDIQUEE M M R,TAJBAKHSH N,et al.Unet++:Redesigning skip connections to exploit multiscale features in image segmentation[J].IEEE Transactions on Medical Imaging,2019,39(6):1856-1867.
[18]FANG B,PAN L,KOU R.Dual Learning-Based SiameseFramework for Change Detection Using Bi-Temporal VHR Optical Remote Sensing Images[J].Remote Sensing,2019,11(11):1292.
[19]CHEN Y,OUYANG X,AGAM G.Mfcnet:End-to-end ap-proach for change detection in images[C]//2018 25th IEEE International Conference on Image Processing.Athens,Greece,IEEE,2018:4008-4012.
[20]HAN X,LEUNG T,JIA Y,et al.MatchNet:Unifying featureand metric learning for patch-based matching[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition.Boston,MA,USA,IEEE 2015:3279-3286.
[21]LONG J,SHELHAMER E,DARRELL T.Fully convolutional networks for semantic segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,39(4):640-651.
[22]CHEN L C,PAPANDREOU G,SCHROFF F,et al.Rethinking atrous convolution for semantic image segmentation[J].arXiv:1706.05587,2017.
[23]SZEGEDY C,LIU W,JIA Y,et al.Going deeper with convolutions[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition.Boston,MA,USA,IEEE 2015:1-9.
[24]CHEN J,YUAN Z,PENG J,et al.DASNet:Dual attentive fully convolutional Siamese networks for change detection in high-resolution satellite images[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2020,14:1194-1206.
[25]CSURKA G,LARLUS D,PERRONNIN F,et al.What is a good evaluation measure for semantic segmentation[C]//2013 Electronic Proceedings of the British Machine Vision Conference.London,British,BMVC,2013.
[26]BOKHOVKIN A,BURNAEV E.Boundary loss for remotesensing imagery semantic segmentation[C]//2019 In International Symposium on Neural Networks.Cham,Germany,Springer,2019:388-401.
[27]LEBEDEV M A,VIZILTER Y V,VYGOLOV O V,et al.Change detection in remote sensing images using conditional adversarial networks[J].International Archives of the Photogrammetry,Remote Sensing & Spatial Information Sciences,2018,42(2):565-571.
[28]CHEN H,SHI Z.A spatial-temporal attention-based method and a new dataset for remote sensing image change detection[J].Remote Sensing,2020,12(10):1662.
[29]DAUDT R C,LE SAUX B,BOULCH A.Fully convolutional siamese networks for change detection[C]//2018 25th IEEE International Conference on Image Processing.Athens,Greece,IEEE 2018:4063-4067.
[30]FANGS,LI K,SHAO J,et al.SNUNet-CD:A densely connected Siamese network for change detection of VHR images[J].IEEE Geoscience and Remote Sensing Letters,2021,19:1-5.
[31]BANDARA W G C,PATEL V M.A transformer-based siamese network for change detection[C]//2022 IEEE International Geoscience and Remote Sensing Symposium(IGARSS 2022).IEEE,2022:207-210.
[32]SHI Q,LIU M,LI S,et al.A deeply supervised attention metric-based network and an open aerial image dataset for remote sensing change detection[J].IEEE Transactions on Geoscience and Remote Sensing,2022,60:1-16.
[33]HAN C,WU C,GUO H,et al.HANet:A hierarchical attention network for change detection with bi-temporal very-high-resolution remote sensing images[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2023,16:3867-3878.
[1] ZHOU Yu, YANG Junling, DANG Kelin. Change Detection in SAR Images Based on Evolutionary Multi-objective Clustering [J]. Computer Science, 2024, 51(9): 140-146.
[2] WANG Qian, HE Lang, WANG Zhanqing, HUANG Kun. Road Extraction Algorithm for Remote Sensing Images Based on Improved DeepLabv3+ [J]. Computer Science, 2024, 51(8): 168-175.
[3] ZHANG Yang, XIA Ying. Object Detection Method with Multi-scale Feature Fusion for Remote Sensing Images [J]. Computer Science, 2024, 51(3): 165-173.
[4] LIU Changxin, WU Ning, HU Lirui, GAO Ba, GAO Xueshan. Recursive Gated Convolution Based Super-resolution Network for Remote Sensing Images [J]. Computer Science, 2024, 51(2): 205-216.
[5] DONG Yan, WEI Minghong, GAO Guangshuai, LIU Zhoufeng, LI Chunlei. Remote Sensing Orineted Object Detection Method Based on Dual-label Assignment [J]. Computer Science, 2024, 51(11A): 240100058-9.
[6] FAN Xuejing, XUE Xiaorong, DU Yichao. Spatiotemporal Fusion Method for Remote Sensing Images Based on Dual Attention Mechanisms [J]. Computer Science, 2024, 51(11A): 240200097-6.
[7] MU Zhengyang, DAI Jianguo, ZHANG Guoshun, HOU Wenqing, CHEN Peipei, CAO Yujuan, XU Miaomiao. Impervious Surface Change Detection in Urban Fringe Areas Based on Multi-scale and Dual-attention Network [J]. Computer Science, 2024, 51(11A): 231200064-8.
[8] YAN Yan, SUI Yi, SI Jianwei. Remote Sensing Image Pan-sharpening Method Based on Generative Adversarial Network [J]. Computer Science, 2023, 50(8): 133-141.
[9] LI Tao, WANG Hairui. Remote Sensing Image Change Detection of Construction Land Based on Siamese AttentionNetwork [J]. Computer Science, 2023, 50(6A): 220500040-5.
[10] LI Shasha, XING Hongjie, LI Gang. Multi-temporal Hyperspectral Anomaly Change Detection Based on Dual Space Conjugate Autoencoder [J]. Computer Science, 2023, 50(12): 175-184.
[11] LI Tao, WANG Hairui, ZHU Guifu. Detection of Farmland Change Based on Unified Attention Fusion Network [J]. Computer Science, 2023, 50(11A): 221100060-6.
[12] WANG Kun-shu, ZHANG Ze-hui, GAO Tie-gang. Reversible Hidden Algorithm for Remote Sensing Images Based on Hachimoji DNA and QR Decomposition [J]. Computer Science, 2022, 49(8): 127-135.
[13] LAN Ling-xiang, CHI Ming-min. Remote Sensing Change Detection Based on Feature Fusion and Attention Network [J]. Computer Science, 2022, 49(6): 193-198.
[14] CHEN Zhi-wen, WANG Kun, ZHOU Guang-yun, WANG Xu, ZHANG Xiao-dan, ZHU Hu-ming. SAR Image Change Detection Method Based on Capsule Network with Weight Pruning [J]. Computer Science, 2021, 48(7): 190-198.
[15] ZHANG Man, LI Jie, ZHU Xin-zhong, SHEN Ji, CHENG Hao-tian. Augmentation Technology of Remote Sensing Dataset Based on Improved DCGAN Algorithm [J]. Computer Science, 2021, 48(6A): 80-84.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!