Computer Science ›› 2023, Vol. 50 ›› Issue (6A): 220500001-6.doi: 10.11896/jsjkx.220500001

• Image Processing & Multimedia Technology • Previous Articles     Next Articles

Semi-supervised Semantic Segmentation for High-resolution Remote Sensing Images Based on DataFusion

GU Yuhang, HAO Jie, CHEN Bing   

  1. School of Computer Science and Technology,Nanjing University of Aeronautics and Astronautics,Nanjing 211106,China
  • Online:2023-06-10 Published:2023-06-12
  • About author:GU Yuhang,born in 1996,postgra-duate.His main research interests include data fusion and semantic segmentation algorithms for high-resolution remote sensing images. HAO Jie,born in 1988,associate professor.Her main research interests include Internet of things,visible light communication and wireless sensing.
  • Supported by:
    National Key Research and Development Program of China (2019YFB2102000).

Abstract: Due to the need for pixel-wise annotation,semantic segmentation usually requires higher labor costs than tasks such as classification and object recognition.Especially in land classification based on high-resolution remote sensing images,complex backgrounds and dense targets make semantic annotation intolerably expensive,which seriously limits the practicability of semantic segmentation algorithms.In addition,although traditional semi/weak supervised learning methods can effectively reduce trai-ning costs,it is difficult to have high application value for the low quality of the segmentation results.In order to solve the above two pain points,this paper proposes a new semi-supervised semantic segmentation model using a self-correcting fusion strategy.By introducing data fusion technology and self-correction mechanism,the dependence of the segmentation model on pixel-wise annotation can be effectively reduced.Our method obtains mean F1-scores of 86.5% and 81.7% on Potsdam and Vaihingen datasets with only 15% pixel-wise annotation.Experimental results show that the proposed model can greatly reduce the cost of training process,and achieve high-quality segmentation results comparable to fully-supervised prediction.

Key words: Remote sensing image, Deep learning, Fully convolutional network, Semantic segmentation, Data fusion, Semi-supervised learning

CLC Number: 

  • TP391
[1]IBRAHIM M S,VAHDAT A,RANJBAR M,et al.Semi-supervised semantic image segmentation with self-correcting networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:12715-12725.
[2]SUN W,WANG R.Fully convolutional networks for semantic segmentation of very high resolution remotely sensed images combined with DSM[J].IEEE Geoscience and Remote Sensing Letters,2018,15(3):474-478.
[3]PIRAMANAYAGAM S,SABER E,SCHWARTZKOPF W,et al.Supervised classification of multisensor remotely sensed images using a deep learning framework[J].Remote Sensing,2018,10(9):1429.
[4]HAZIRBAS C,MA L,DOMOKOS C,et al.Fusenet:Incorpora-ting depth into semantic segmentation via fusion-based cnn architecture[C]//Asian Conference on Computer Vision.Springer,2016:213-228.
[5]LONG J,SHELHAMER E,DARRELL T.Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:3431-3440.
[6]ISPRS Semantic Labeling Contest(2D)[OL].https://www2.isprs.org/commissions/comm2/wg4/benchmark/semantic-labe-ling/(accessed on 16 October 2021).
[7]DAI J,HE K,SUN J.Boxsup:Exploiting bounding boxes to supervise convolutional networks for semantic segmentation[C]//Proceedings of the IEEE International Conference on Computer Vision.2015:1635-1643.
[8]SONG C,HUANG Y,OUYANG W,et al.Box-driven class-wise region masking and filling rate guided loss for weakly supervised semantic segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:3136-3145.
[9]LIN D,DAI J,JIA J,et al.Scribblesup:Scribble-supervised convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:3159-3167.
[10]ROTHER C,KOLMOGOROV V,BLAKE A."GrabCut" interactive foreground extraction using iterated graph cuts[J].ACM Transactions on Graphics(TOG),2004,23(3):309-314.
[11]ZHOU B,KHOSLA A,LAPEDRIZA A,et al.Learning deepfeatures for discriminative localization[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:2921-2929.
[12]SZEGEDY C,VANHOUCKE V,IOFFE S,et al.Rethinking the inception architecture for computer vision[C]//Proceedings of the IEEE conference on computer vision and pattern recognition.2016:2818-2826.
[13]WEI Y,XIAO H,SHI H,et al.Revisiting dilated convolution:A simple approach for weakly-and semi-supervised semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:7268-7277.
[14]MOREIRA A,PRATS-IRAOLA P,YOUNIS M,et al.A tutorial on synthetic aperture radar[J].IEEE Geoscience and remote sensing magazine,2013,1(1):6-43.
[15]LI S,WANG Y,CHEN P,et al.Spatiotemporal fuzzy clustering strategy for urban expansion monitoring based on time series of pixel-level optical and SAR images[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2017,10(5):1769-1779.
[16]HAAS J,BAN Y.Sentinel-1A SAR and sentinel-2A MSI data fusion for urban ecosystem service mapping[J].Remote Sensing Applications:Society and Environment,2017,8:41-53.
[17]SOHN G,DOWMAN I.Data fusion of high-resolution satellite imagery and LiDAR data for automatic building extraction[J].ISPRS Journal of Photogrammetry and Remote Sensing,2007,62(1):43-63.
[18]AWRANGJEB M,RAVANBAKHSH M,FRASER C S.Automatic detection of residential buildings using LIDAR data and multispectral imagery[J].ISPRS Journal of Photogrammetry and Remote Sensing,2010,65(5):457-467.
[19]DECHESNE C,MALLET C,LE BRIS A,et al.Semantic seg-mentation of forest stands of pure species combining airborne lidar data and very high resolution multispectral imagery[J].ISPRS Journal of Photogrammetry and Remote Sensing,2017,126:129-145.
[20]CHEN L C,ZHU Y,PAPANDREOU G,et al.Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of the European conference on computer vision(ECCV).2018:801-818.
[21]GU Y,HAO J,CHEN B,et al.Top-Down Pyramid Fusion Network for High-Resolution Remote Sensing Semantic Segmentation[J].Remote Sensing,2021,13(20):4159.
[22]ISPRS Semantic Labeling Contest(2D) Results[OL].https://www2.isprs.org/commissions/comm2/wg4/results/potsdam-2d-semantic-labeling/(accessed on 16 October 2021).
[23]VOLPI M,TUIA D.Dense semantic labeling of subdecimeterresolution images with convolutional neural networks[J].IEEE Transactions on Geoscience and Remote Sensing,2016,55(2):881-893.
[24]KEMKER R,SALVAGGIO C,KANAN C.Algorithms for semantic segmentation of multispectral remote sensing imagery using deep learning[J].ISPRS Journal of Photogrammetry and Remote Sensing,2018,145:60-77.
[25]ISPRS Semantic Labeling Contest(2D) Results[OL].https://www2.isprs.org/commissions/comm2/wg4/results/vaihingen-2d-semantic-labeling-contest/(accessed on 16 October 2021).
[26]ACUNA D,KAR A,FIDLER S.Devil is in the edges:Learning semantic boundaries from noisy annotations[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:11075-11083.
[27]YU Z,LIU W,ZOU Y,et al.Simultaneous edge alignment and learning[C]//Proceedings of the European Conference on Computer Vision(ECCV).2018:388-404.
[1] WANG Yu, WANG Zuchao, PAN Rui. Survey of DGA Domain Name Detection Based on Character Feature [J]. Computer Science, 2023, 50(8): 251-259.
[2] ZHANG Yian, YANG Ying, REN Gang, WANG Gang. Study on Multimodal Online Reviews Helpfulness Prediction Based on Attention Mechanism [J]. Computer Science, 2023, 50(8): 37-44.
[3] SONG Xinyang, YAN Zhiyuan, SUN Muyi, DAI Linlin, LI Qi, SUN Zhenan. Review of Talking Face Generation [J]. Computer Science, 2023, 50(8): 68-78.
[4] WANG Xu, WU Yanxia, ZHANG Xue, HONG Ruize, LI Guangsheng. Survey of Rotating Object Detection Research in Computer Vision [J]. Computer Science, 2023, 50(8): 79-92.
[5] ZHOU Ziyi, XIONG Hailing. Image Captioning Optimization Strategy Based on Deep Learning [J]. Computer Science, 2023, 50(8): 99-110.
[6] ZHANG Xiao, DONG Hongbin. Lightweight Multi-view Stereo Integrating Coarse Cost Volume and Bilateral Grid [J]. Computer Science, 2023, 50(8): 125-132.
[7] YAN Yan, SUI Yi, SI Jianwei. Remote Sensing Image Pan-sharpening Method Based on Generative Adversarial Network [J]. Computer Science, 2023, 50(8): 133-141.
[8] LI Kun, GUO Wei, ZHANG Fan, DU Jiayu, YANG Meiyue. Adversarial Malware Generation Method Based on Genetic Algorithm [J]. Computer Science, 2023, 50(7): 325-331.
[9] WANG Mingxia, XIONG Yun. Disease Diagnosis Prediction Algorithm Based on Contrastive Learning [J]. Computer Science, 2023, 50(7): 46-52.
[10] LI Hui, LI Wengen, GUAN Jihong. Dually Encoded Semi-supervised Anomaly Detection [J]. Computer Science, 2023, 50(7): 53-59.
[11] SHEN Zhehui, WANG Kailai, KONG Xiangjie. Exploring Station Spatio-Temporal Mobility Pattern:A Short and Long-term Traffic Prediction Framework [J]. Computer Science, 2023, 50(7): 98-106.
[12] HUO Weile, JING Tao, REN Shuang. Review of 3D Object Detection for Autonomous Driving [J]. Computer Science, 2023, 50(7): 107-118.
[13] ZHOU Bo, JIANG Peifeng, DUAN Chang, LUO Yuetong. Study on Single Background Object Detection Oriented Improved-RetinaNet Model and Its Application [J]. Computer Science, 2023, 50(7): 137-142.
[14] MAO Huihui, ZHAO Xiaole, DU Shengdong, TENG Fei, LI Tianrui. Short-term Subway Passenger Flow Forecasting Based on Graphical Embedding of Temporal Knowledge [J]. Computer Science, 2023, 50(7): 213-220.
[15] LI Yuqiang, LI Linfeng, ZHU Hao, HOU Mengshu. Deep Learning-based Algorithm for Active IPv6 Address Prediction [J]. Computer Science, 2023, 50(7): 261-269.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!