Computer Science ›› 2021, Vol. 48 ›› Issue (8): 162-168.doi: 10.11896/jsjkx.200700182

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Remote Sensing Image Semantic Segmentation Method Based on U-Net Feature Fusion Optimization Strategy

WANG Shi-yun, YANG Fan   

  1. School of Electronic and Information Engineering,Hebei University of Technology,Tianjin 300401,China
  • Received:2020-07-28 Revised:2020-09-19 Published:2021-08-10
  • About author:WANG Shi-yun,born in 1994,postgra-duate.Her main research interests include intelligent information processing and so on.( Fan,born in 1966,Ph.D,professor,Ph.D supervisor.His main research interests include computer vision inspection technology,image processing and pattern recognition research.
  • Supported by:
    National Key R&D Program Intelligent Robot Special Project (2019YFB1312102) and Natural Science Foundation of Hebei Province (F2019202364).

Abstract: Due to the high spatial resolution of high-resolution remote sensing images,rich ground objects information,high complexity,uneven distribution of target categories and different sizes of various ground objects,it is difficult to improve the segmentation accuracy.In order to improve the semantic segmentation accuracy of remote sensing images and solve the problem that U-Net model is limited when combining deep semantic information and shallow position information,a semantic segmentation me-thod of remote sensing images based on U-Net feature fusion optimization strategy is proposed.This method adopts the encoder-decoder structure based on U-Net network.In the feature extraction part of the network,the encoder structure of U-Net model is used to extract the feature information of multiple layers.In the feature fusion part,the jump connection structure of U-Net is retained,and at the same time,the feature fusion optimization strategy proposed in this paper is used to realize the fusion-optimization-refusion of high-level semantic features and low-level location features.In addition,the feature fusion optimization strategy uses dilated convolution to get more global features,and uses Sub-Pixel convolutional layer instead of traditional transposed convolution to achieve adaptive upsampling.This method is validated on the Potsdam dataset and Vaihingen dataset of ISPRS.The three evaluation indexes,overall classification accuracy,Kappa coefficient and mIoU in the verification are 86.2%,0.82,0.77 on Potsdam dataset,and 84.5%,0.79,0.69 on Vaihingen dataset.Compared with the traditional U-Net model,the three evaluation indicators are increased by 5.8%,8%,8% on Potsdam dataset,and 3.5%,4%,11% on Vaihingen dataset.Experimental results show that the remote sensing image semantic segmentation method based on the U-Net feature fusion optimization strategy has achieved good semantic segmentation effects on both the Potsdam dataset and the Vaihingen dataset,which can improve the accuracy of semantic segmentation of remote sensing images.

Key words: Deep learning, Dilated convolution, Feature fusion, Remote sensing image, Semantic segmentation

CLC Number: 

  • TP391
[1]WANG B,FAN D L.A Summary of the Research Progress of Deep Learning in Remote Sensing Image Classification and Re-cognition[J].Bulletin of Surveying and Mapping,2019,503(2):108-111,145.
[2]QIN Y Q,CHI M M.High-resolution remote sensing image semantic segmentation method combined with scene classification data[J].Computer Applications and Software,2020,37(06):126-129,134.
[3]WANG E D,QI K,LI X P,et al.Semantic segmentation method of remote sensing image based on neural network[J].Acta Optica Sinica,2019,39(12):93-104.
[4]LONG J,SHELHAMER E,DARRELL T.Fully convolutional networks for semantic segmentation[C]//The IEEE Conference on Computer Vision and Pattern Recognition.Boston,USA,2015:3431-3440.
[5]YU F,KOLTUN V.Multi-Scale Context Aggregation by Dila-ted Convolutions[C]//International Conference on Learning Representations.San Juan,Puerto Rico,2016.
[6]CHEN L C,PAPANDEROU G,KOKKINOS I,et al.DeepLab:Semantic Image Segmentation with Deep Convolutional Nets,Atrous Convolution,and Fully Connected CRFS[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2016,40(4):834-848.
[7]RONNEBERGER O,FISCHER P,BROX T,et al.U-net:Con-volutional networks for biomedical image segmentation[J].Medical Image Computing and Computer Assisted Intervention,2015,28(4):234-241.
[8]YUAN J Y.Automatic building extraction in aerial scenes using convolutional networks[J].arXiv:1602.06564,2016.
[9]SU J M,YANG L X,JING W P.Semantic segmentation method of high-resolution remote sensing image based on U-Net[J].Computer Engineering and Applications,2019,55(7):207-213.
[10]BERMAN M,TRIKI A R,BLASCHKO M B.The Lovász-softmax loss:a tractable surrogate for the optimization of the intersection-over-union measure in neural networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City:UT,2018:4413-4421.
[11]SHI W Z,CABALLERO J,HUSZAR F,et al.Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Las Vegas,NV,2016:1874-1883.
[12]MAGGIORI E,TARABALKA Y,CHARPIAT G,et al.High-resolution aerial image labeling with convolutional neural networks[C]//IEEE Transactions on Geoscience and Remote Sensing.2017:7092-7103.
[13]ZHOU J Y,ZHAO Y M.Overview of Convolutiotnal NeuralNetworks in Image Classification and Target Detection[J].Computer Engineering and Applications,2017,53(13):34-41.
[14]PASCANU R,MIKOLOV T,BENGIO Y.On the difficulty of training recurrent neural networks[C]//Proceedings of the 30th International Conference on Machine Learning(CML2013).Atlanta,GA,USA,2013:1310-1318.
[15]IOFFE S,SZEGEDY C.Batch normalization:Accelerating deep network training by reducing internal covariate shift[J].arXiv:1502.03167v3,2015.
[16]XU Z J,YANG X B,HE L M,et al.Multiscale remote sensing semantic segmentation network[J/OL].Computer Engineering and Applications:1-9[2020-07-18].
[1] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[2] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[3] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[4] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[5] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[6] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[7] WANG Kun-shu, ZHANG Ze-hui, GAO Tie-gang. Reversible Hidden Algorithm for Remote Sensing Images Based on Hachimoji DNA and QR Decomposition [J]. Computer Science, 2022, 49(8): 127-135.
[8] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[9] HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[10] ZHANG Ying-tao, ZHANG Jie, ZHANG Rui, ZHANG Wen-qiang. Photorealistic Style Transfer Guided by Global Information [J]. Computer Science, 2022, 49(7): 100-105.
[11] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[12] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[13] ZHOU Hui, SHI Hao-chen, TU Yao-feng, HUANG Sheng-jun. Robust Deep Neural Network Learning Based on Active Sampling [J]. Computer Science, 2022, 49(7): 164-169.
[14] SU Dan-ning, CAO Gui-tao, WANG Yan-nan, WANG Hong, REN He. Survey of Deep Learning for Radar Emitter Identification Based on Small Sample [J]. Computer Science, 2022, 49(7): 226-235.
[15] ZHU Wen-tao, LAN Xian-chao, LUO Huan-lin, YUE Bing, WANG Yang. Remote Sensing Aircraft Target Detection Based on Improved Faster R-CNN [J]. Computer Science, 2022, 49(6A): 378-383.
Full text



No Suggested Reading articles found!