Computer Science ›› 2021, Vol. 48 ›› Issue (8): 162-168.doi: 10.11896/jsjkx.200700182

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Remote Sensing Image Semantic Segmentation Method Based on U-Net Feature Fusion Optimization Strategy

WANG Shi-yun, YANG Fan   

  1. School of Electronic and Information Engineering,Hebei University of Technology,Tianjin 300401,China
  • Received:2020-07-28 Revised:2020-09-19 Published:2021-08-10
  • About author:WANG Shi-yun,born in 1994,postgra-duate.Her main research interests include intelligent information processing and so on.( Fan,born in 1966,Ph.D,professor,Ph.D supervisor.His main research interests include computer vision inspection technology,image processing and pattern recognition research.
  • Supported by:
    National Key R&D Program Intelligent Robot Special Project (2019YFB1312102) and Natural Science Foundation of Hebei Province (F2019202364).

Abstract: Due to the high spatial resolution of high-resolution remote sensing images,rich ground objects information,high complexity,uneven distribution of target categories and different sizes of various ground objects,it is difficult to improve the segmentation accuracy.In order to improve the semantic segmentation accuracy of remote sensing images and solve the problem that U-Net model is limited when combining deep semantic information and shallow position information,a semantic segmentation me-thod of remote sensing images based on U-Net feature fusion optimization strategy is proposed.This method adopts the encoder-decoder structure based on U-Net network.In the feature extraction part of the network,the encoder structure of U-Net model is used to extract the feature information of multiple layers.In the feature fusion part,the jump connection structure of U-Net is retained,and at the same time,the feature fusion optimization strategy proposed in this paper is used to realize the fusion-optimization-refusion of high-level semantic features and low-level location features.In addition,the feature fusion optimization strategy uses dilated convolution to get more global features,and uses Sub-Pixel convolutional layer instead of traditional transposed convolution to achieve adaptive upsampling.This method is validated on the Potsdam dataset and Vaihingen dataset of ISPRS.The three evaluation indexes,overall classification accuracy,Kappa coefficient and mIoU in the verification are 86.2%,0.82,0.77 on Potsdam dataset,and 84.5%,0.79,0.69 on Vaihingen dataset.Compared with the traditional U-Net model,the three evaluation indicators are increased by 5.8%,8%,8% on Potsdam dataset,and 3.5%,4%,11% on Vaihingen dataset.Experimental results show that the remote sensing image semantic segmentation method based on the U-Net feature fusion optimization strategy has achieved good semantic segmentation effects on both the Potsdam dataset and the Vaihingen dataset,which can improve the accuracy of semantic segmentation of remote sensing images.

Key words: Deep learning, Feature fusion, Remote sensing image, Dilated convolution, Semantic segmentation

CLC Number: 

  • TP391
[1]WANG B,FAN D L.A Summary of the Research Progress of Deep Learning in Remote Sensing Image Classification and Re-cognition[J].Bulletin of Surveying and Mapping,2019,503(2):108-111,145.
[2]QIN Y Q,CHI M M.High-resolution remote sensing image semantic segmentation method combined with scene classification data[J].Computer Applications and Software,2020,37(06):126-129,134.
[3]WANG E D,QI K,LI X P,et al.Semantic segmentation method of remote sensing image based on neural network[J].Acta Optica Sinica,2019,39(12):93-104.
[4]LONG J,SHELHAMER E,DARRELL T.Fully convolutional networks for semantic segmentation[C]//The IEEE Conference on Computer Vision and Pattern Recognition.Boston,USA,2015:3431-3440.
[5]YU F,KOLTUN V.Multi-Scale Context Aggregation by Dila-ted Convolutions[C]//International Conference on Learning Representations.San Juan,Puerto Rico,2016.
[6]CHEN L C,PAPANDEROU G,KOKKINOS I,et al.DeepLab:Semantic Image Segmentation with Deep Convolutional Nets,Atrous Convolution,and Fully Connected CRFS[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2016,40(4):834-848.
[7]RONNEBERGER O,FISCHER P,BROX T,et al.U-net:Con-volutional networks for biomedical image segmentation[J].Medical Image Computing and Computer Assisted Intervention,2015,28(4):234-241.
[8]YUAN J Y.Automatic building extraction in aerial scenes using convolutional networks[J].arXiv:1602.06564,2016.
[9]SU J M,YANG L X,JING W P.Semantic segmentation method of high-resolution remote sensing image based on U-Net[J].Computer Engineering and Applications,2019,55(7):207-213.
[10]BERMAN M,TRIKI A R,BLASCHKO M B.The Lovász-softmax loss:a tractable surrogate for the optimization of the intersection-over-union measure in neural networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City:UT,2018:4413-4421.
[11]SHI W Z,CABALLERO J,HUSZAR F,et al.Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Las Vegas,NV,2016:1874-1883.
[12]MAGGIORI E,TARABALKA Y,CHARPIAT G,et al.High-resolution aerial image labeling with convolutional neural networks[C]//IEEE Transactions on Geoscience and Remote Sensing.2017:7092-7103.
[13]ZHOU J Y,ZHAO Y M.Overview of Convolutiotnal NeuralNetworks in Image Classification and Target Detection[J].Computer Engineering and Applications,2017,53(13):34-41.
[14]PASCANU R,MIKOLOV T,BENGIO Y.On the difficulty of training recurrent neural networks[C]//Proceedings of the 30th International Conference on Machine Learning(CML2013).Atlanta,GA,USA,2013:1310-1318.
[15]IOFFE S,SZEGEDY C.Batch normalization:Accelerating deep network training by reducing internal covariate shift[J].arXiv:1502.03167v3,2015.
[16]XU Z J,YANG X B,HE L M,et al.Multiscale remote sensing semantic segmentation network[J/OL].Computer Engineering and Applications:1-9[2020-07-18].
[1] FENG Xia, HU Zhi-yi, LIU Cai-hua. Survey of Research Progress on Cross-modal Retrieval [J]. Computer Science, 2021, 48(8): 13-23.
[2] WANG Li-mei, ZHU Xu-guang, WANG De-jia, ZHANG Yong, XING Chun-xiao. Study on Judicial Data Classification Method Based on Natural Language Processing Technologies [J]. Computer Science, 2021, 48(8): 80-85.
[3] YE Zhong-yu, WU Meng-lin. Choroidal Neovascularization Segmentation Combining Temporal Supervision and Attention Mechanism [J]. Computer Science, 2021, 48(8): 118-124.
[4] GUO Lin, LI Chen, CHEN Chen, ZHAO Rui, FAN Shi-lin, XU Xing-yu. Image Super-resolution Reconstruction Using Recursive ResidualNetwork Based on ChannelAttention [J]. Computer Science, 2021, 48(8): 139-144.
[5] LIU Shuai, RUI Ting, HU Yu-cheng, YANG Cheng-song, WANG Dong. Monocular Visual Odometer Based on Deep Learning SuperGlue Algorithm [J]. Computer Science, 2021, 48(8): 157-161.
[6] TIAN Song-wang, LIN Su-zhen, YANG Bo. Multi-band Image Self-supervised Fusion Method Based on Multi-discriminator [J]. Computer Science, 2021, 48(8): 185-190.
[7] PAN Xiao-qin, LU Tian-liang, DU Yan-hui, TONG Xin. Overview of Speech Synthesis and Voice Conversion Technology Based on Deep Learning [J]. Computer Science, 2021, 48(8): 200-208.
[8] TANG Shi-zheng, ZHANG Yan-feng. DragDL:An Easy-to-Use Graphical DL Model Construction System [J]. Computer Science, 2021, 48(8): 220-225.
[9] ZHANG Jin, DUAN Li-guo, LI Ai-ping, HAO Xiao-yan. Fine-grained Sentiment Analysis Based on Combination of Attention and Gated Mechanism [J]. Computer Science, 2021, 48(8): 226-233.
[10] LIU Wen-yang, GUO Yan-bu, LI Wei-hua. Identifying Essential Proteins by Hybrid Deep Learning Model [J]. Computer Science, 2021, 48(8): 240-245.
[11] WANG Chao, WEI Xiang-lin, TIAN Qing, JIAO Xiang, WEI Nan, DUAN Qiang. Feature Gradient-based Adversarial Attack on Modulation Recognition-oriented Deep Neural Networks [J]. Computer Science, 2021, 48(7): 25-32.
[12] YANG Yang, CHEN Wei, ZHANG Dan-yi, WANG Dan-ni, SONG Shuang. Adversarial Attacks Threatened Network Traffic Classification Based on CNN [J]. Computer Science, 2021, 48(7): 55-61.
[13] BAO Yu-xuan, LU Tian-liang, DU Yan-hui, SHI Da. Deepfake Videos Detection Method Based on i_ResNet34 Model and Data Augmentation [J]. Computer Science, 2021, 48(7): 77-85.
[14] SANG Chun-yan, XU Wen, JIA Chao-long, WEN Jun-hao. Prediction of Evolution Trend of Online Public Opinion Events Based on Attention Mechanism in Social Networks [J]. Computer Science, 2021, 48(7): 118-123.
[15] XU Hao, LIU Yue-lei. UAV Sound Recognition Algorithm Based on Deep Learning [J]. Computer Science, 2021, 48(7): 225-232.
Full text



[1] YANG Yu-qi, ZHANG Guo-an and JIN Xi-long. Dual-cluster-head Routing Protocol Based on Vehicle Density in VANETs[J]. Computer Science, 2018, 45(4): 126 -130 .
[2] WANG Zhen-wu, LV Xiao-hua and HAN Xiao-hui. Survey of Terrain LOD Technology Based on Quadtree Segmentation[J]. Computer Science, 2018, 45(4): 34 -45 .
[3] LUO Jian-zhen,CAI Jun ,LIU Yan,ZHAO Hui-min. Caching and Replacing Strategy in Information-centric Network Based on Content Popularity and Community Importance[J]. Computer Science, 2018, 45(7): 116 -121 .
[4] LIU Xiao, WANG Xiao-guo. Probabilistic Graphical Model Based Approach for Bank Telecommunication Fraud Detection[J]. Computer Science, 2018, 45(7): 122 -128 .
[5] WANG Rong, LIU Zun-ren, JI Jun. Fast Attribute Reduction Algorithm Based on Importance of Voting Attribute[J]. Computer Science, 2018, 45(7): 197 -201 .
[6] HE Xiao-jun, WU Meng-lin, FAN Wen, YUAN Song-tao, CHEN Qiang. SD-OCT CSC NRD Region Segmentation Based on Region Restricted 3D Region Growing[J]. Computer Science, 2018, 45(6A): 187 -192 .
[7] PENG Yan,WU Zhao-qiang, ZHANG Jing-kuo, CHEN Run-xue. Improved Difference Algorithm and It’s Application in QRS Detection[J]. Computer Science, 2018, 45(6A): 588 -590 .
[8] ZHANG Gang, GAO Jun-peng, LI Hong-wei. Research on Stochastic Resonance Characteristics of Cascaded Three-steady-state and Its Application[J]. Computer Science, 2018, 45(9): 146 -151 .
[9] GAO Peng, LIU Yun-jiang, GAO Wei-ting, LI Man, CHEN Juan. Double Thresholds DMM Cooperative Spectrum Sensing Algorithm Based on Credibility[J]. Computer Science, 2018, 45(9): 166 -170 .
[10] ZHOU Yan-fang, ZHOU Gang, LU Zhong-lei. Approach of Stance Detection in Micro-blog Based on Transfer Learning and Multi-representation[J]. Computer Science, 2018, 45(9): 243 -247 .