计算机科学 ›› 2023, Vol. 50 ›› Issue (6A): 220500001-6.doi: 10.11896/jsjkx.220500001

• 图像处理&多媒体技术 • 上一篇    下一篇

基于数据融合的半监督高分遥感影像语义分割

顾宇航, 郝洁, 陈兵   

  1. 南京航空航天大学计算机科学与技术学院 南京 211106
  • 出版日期:2023-06-10 发布日期:2023-06-12
  • 通讯作者: 郝洁(haojie@nuaa.edu.cn)
  • 作者简介:(melon@nuaa.edu.cn)
  • 基金资助:
    国家重点研发计划(2019YFB2102000)

Semi-supervised Semantic Segmentation for High-resolution Remote Sensing Images Based on DataFusion

GU Yuhang, HAO Jie, CHEN Bing   

  1. School of Computer Science and Technology,Nanjing University of Aeronautics and Astronautics,Nanjing 211106,China
  • Online:2023-06-10 Published:2023-06-12
  • About author:GU Yuhang,born in 1996,postgra-duate.His main research interests include data fusion and semantic segmentation algorithms for high-resolution remote sensing images. HAO Jie,born in 1988,associate professor.Her main research interests include Internet of things,visible light communication and wireless sensing.
  • Supported by:
    National Key Research and Development Program of China (2019YFB2102000).

摘要: 由于需要进行像素级标注,语义分割通常比分类以及目标识别等任务需要更高的人工成本,尤其在基于高分遥感影像的土地分类应用中,因其背景复杂、目标密集,进行语义标注的成本更为高昂,严重限制了该技术在智能遥感领域的发展。此外,尽管传统半/弱监督学习方法能够有效降低训练成本,但通常其分割结果的质量较低,很难具备应用价值。针对以上两个问题,文中提出了一种采用半监督自校正融合策略的语义分割模型。通过引入数据融合技术以及自校正策略,有效地降低了分割模型对强标注的依赖性。该模型在仅使用15%强监督信息的前提下,在波茨坦以及韦兴根数据集上分别获得了86.5%和81.7%的平均F1分数。实验结果表明,所提方法在大幅降低语义分割训练成本的同时,能够获得与全监督模型相竞争的高质量分割结果。

关键词: 遥感图像, 深度学习, 全卷积神经网络, 语义分割, 数据融合, 半监督学习

Abstract: Due to the need for pixel-wise annotation,semantic segmentation usually requires higher labor costs than tasks such as classification and object recognition.Especially in land classification based on high-resolution remote sensing images,complex backgrounds and dense targets make semantic annotation intolerably expensive,which seriously limits the practicability of semantic segmentation algorithms.In addition,although traditional semi/weak supervised learning methods can effectively reduce trai-ning costs,it is difficult to have high application value for the low quality of the segmentation results.In order to solve the above two pain points,this paper proposes a new semi-supervised semantic segmentation model using a self-correcting fusion strategy.By introducing data fusion technology and self-correction mechanism,the dependence of the segmentation model on pixel-wise annotation can be effectively reduced.Our method obtains mean F1-scores of 86.5% and 81.7% on Potsdam and Vaihingen datasets with only 15% pixel-wise annotation.Experimental results show that the proposed model can greatly reduce the cost of training process,and achieve high-quality segmentation results comparable to fully-supervised prediction.

Key words: Remote sensing image, Deep learning, Fully convolutional network, Semantic segmentation, Data fusion, Semi-supervised learning

中图分类号: 

  • TP391
[1]IBRAHIM M S,VAHDAT A,RANJBAR M,et al.Semi-supervised semantic image segmentation with self-correcting networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:12715-12725.
[2]SUN W,WANG R.Fully convolutional networks for semantic segmentation of very high resolution remotely sensed images combined with DSM[J].IEEE Geoscience and Remote Sensing Letters,2018,15(3):474-478.
[3]PIRAMANAYAGAM S,SABER E,SCHWARTZKOPF W,et al.Supervised classification of multisensor remotely sensed images using a deep learning framework[J].Remote Sensing,2018,10(9):1429.
[4]HAZIRBAS C,MA L,DOMOKOS C,et al.Fusenet:Incorpora-ting depth into semantic segmentation via fusion-based cnn architecture[C]//Asian Conference on Computer Vision.Springer,2016:213-228.
[5]LONG J,SHELHAMER E,DARRELL T.Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:3431-3440.
[6]ISPRS Semantic Labeling Contest(2D)[OL].https://www2.isprs.org/commissions/comm2/wg4/benchmark/semantic-labe-ling/(accessed on 16 October 2021).
[7]DAI J,HE K,SUN J.Boxsup:Exploiting bounding boxes to supervise convolutional networks for semantic segmentation[C]//Proceedings of the IEEE International Conference on Computer Vision.2015:1635-1643.
[8]SONG C,HUANG Y,OUYANG W,et al.Box-driven class-wise region masking and filling rate guided loss for weakly supervised semantic segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:3136-3145.
[9]LIN D,DAI J,JIA J,et al.Scribblesup:Scribble-supervised convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:3159-3167.
[10]ROTHER C,KOLMOGOROV V,BLAKE A."GrabCut" interactive foreground extraction using iterated graph cuts[J].ACM Transactions on Graphics(TOG),2004,23(3):309-314.
[11]ZHOU B,KHOSLA A,LAPEDRIZA A,et al.Learning deepfeatures for discriminative localization[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:2921-2929.
[12]SZEGEDY C,VANHOUCKE V,IOFFE S,et al.Rethinking the inception architecture for computer vision[C]//Proceedings of the IEEE conference on computer vision and pattern recognition.2016:2818-2826.
[13]WEI Y,XIAO H,SHI H,et al.Revisiting dilated convolution:A simple approach for weakly-and semi-supervised semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:7268-7277.
[14]MOREIRA A,PRATS-IRAOLA P,YOUNIS M,et al.A tutorial on synthetic aperture radar[J].IEEE Geoscience and remote sensing magazine,2013,1(1):6-43.
[15]LI S,WANG Y,CHEN P,et al.Spatiotemporal fuzzy clustering strategy for urban expansion monitoring based on time series of pixel-level optical and SAR images[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2017,10(5):1769-1779.
[16]HAAS J,BAN Y.Sentinel-1A SAR and sentinel-2A MSI data fusion for urban ecosystem service mapping[J].Remote Sensing Applications:Society and Environment,2017,8:41-53.
[17]SOHN G,DOWMAN I.Data fusion of high-resolution satellite imagery and LiDAR data for automatic building extraction[J].ISPRS Journal of Photogrammetry and Remote Sensing,2007,62(1):43-63.
[18]AWRANGJEB M,RAVANBAKHSH M,FRASER C S.Automatic detection of residential buildings using LIDAR data and multispectral imagery[J].ISPRS Journal of Photogrammetry and Remote Sensing,2010,65(5):457-467.
[19]DECHESNE C,MALLET C,LE BRIS A,et al.Semantic seg-mentation of forest stands of pure species combining airborne lidar data and very high resolution multispectral imagery[J].ISPRS Journal of Photogrammetry and Remote Sensing,2017,126:129-145.
[20]CHEN L C,ZHU Y,PAPANDREOU G,et al.Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of the European conference on computer vision(ECCV).2018:801-818.
[21]GU Y,HAO J,CHEN B,et al.Top-Down Pyramid Fusion Network for High-Resolution Remote Sensing Semantic Segmentation[J].Remote Sensing,2021,13(20):4159.
[22]ISPRS Semantic Labeling Contest(2D) Results[OL].https://www2.isprs.org/commissions/comm2/wg4/results/potsdam-2d-semantic-labeling/(accessed on 16 October 2021).
[23]VOLPI M,TUIA D.Dense semantic labeling of subdecimeterresolution images with convolutional neural networks[J].IEEE Transactions on Geoscience and Remote Sensing,2016,55(2):881-893.
[24]KEMKER R,SALVAGGIO C,KANAN C.Algorithms for semantic segmentation of multispectral remote sensing imagery using deep learning[J].ISPRS Journal of Photogrammetry and Remote Sensing,2018,145:60-77.
[25]ISPRS Semantic Labeling Contest(2D) Results[OL].https://www2.isprs.org/commissions/comm2/wg4/results/vaihingen-2d-semantic-labeling-contest/(accessed on 16 October 2021).
[26]ACUNA D,KAR A,FIDLER S.Devil is in the edges:Learning semantic boundaries from noisy annotations[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:11075-11083.
[27]YU Z,LIU W,ZOU Y,et al.Simultaneous edge alignment and learning[C]//Proceedings of the European Conference on Computer Vision(ECCV).2018:388-404.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!