Computer Science ›› 2025, Vol. 52 ›› Issue (5): 212-219.doi: 10.11896/jsjkx.240300137

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Improved U-Net Multi-scale Feature Fusion Semantic Segmentation Network for RemoteSensing Images

JIANG Wenwen, XIA Ying   

  1. College of Computer Science and Technology,Chongqing University of Posts and Telecommunications,Chongqing 400065,China
    Key Laboratory of Tourism Multisource Data Perception and Decision Technology,Ministry of Culture and Tourism,Chongqing 400065,China
  • Received:2024-03-20 Revised:2024-07-22 Online:2025-05-15 Published:2025-05-12
  • About author:JIANG Wenwen,born in 2000,postgraduate.Her main research interests include intelligent analysis of remote sensing images.
    XIA Ying,born in 1972,Ph.D,professor,Ph.D supervisor,is a senior member of CCF(No.10248S).Her main research interests include spatiotemporal big data and cross-media retrieval.
  • Supported by:
    Chongqing Municipal Education Commission Cooperation Projects(HZ2021008) and Key Laboratory Funding Project of Cultural and Tourism Department(E020H2023005).

Abstract: High spatial resolution of remote sensing images,the large scale differences of different types of objects,and the imba-lance of categories are the main challenges faced by accurate semantic segmentation tasks.In order to improve the accuracy of semantic segmentation of remote sensing images,this paper proposes an improved U-Net multi-scale feature fusion semantic segmentation network for remote sensing image(Multi-scale Feature Fusion Network,MFFNet).The network is based on the U-Net network and includes a dynamic feature fusion module and a gated attention convolution mix module.Among them,the dynamic feature fusion module replaces the skip connection and improves the feature fusion method of the upsampling layer and the downsampling layer to avoid information loss caused by feature fusion,while improving the fusion effect of shallow features and deep features.Gated attention convolution mix module integrates self-attention,convolution,and gating mechanisms to better capture both local and global information.Comparative experiments and ablation experiments are carried out on Potsdam and Vaihingen.The results show that the mIoU of MFFNet on the two datasets reached 76.95% and 72.93% respectively,effectively improving the semantic segmentation accuracy of remote sensing images.

Key words: Semantic segmentation, Remote sensing images, Attention mechanism, Feature fusion, Gating mechanism

CLC Number: 

  • TP391
[1]WANG J,DING J,RAN S,et al.Automatic Pear Extractionfrom High-Resolution Images by a Visual Attention Mechanism Network[J].Remote Sensing,2023,15(13):3283-3298.
[2]MA Y.Research Review of Image Semantic Segmentation Methods in High-Resolution Remote Sensing Image Interpretation[J].Journal of Frontiers of ComputerScience and Technology,2023,17(7):1526-1548.
[3]ZHAN Z Y,AN Y J,C C W.Image Threshold Segmentation Algorithms and Comparative Research[J].Information and Communication,2017(4):86-89.
[4]LIANG Z X,WANG X B,HE T,et al.Research and implementation of instance segmentation and edge optimization algorithms[J].Journal of Graphics,2020,41(6):939-946.
[5]ADAMS R,BISCHOF L.Seeded region growing[J].IEEETransactions on Pattern Analysis and Machine Intelligence,1994,16(6):641-647.
[6]LONG J,SHELHAMER E,DARRELL T.Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:3431-3440.
[7]RONNEBERGER O,FISCHER P,BROX T.U-Net:Convolu-tional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Computer-assisted Intervention.Cham:Springer,2015:234-241.
[8]BADRINARAYANAN V,KENDALL A,CIPOLLA R.Segnet:A deep convolutional encoder-decoder architecture for image segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(12):2481-2495.
[9]ZHAO H,SHI J,QI X,et al.Pyramid scene parsing network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:2881-2890.
[10]CHEN L C,PAPANDREOU G,KOKKINOS I,et al.Deeplab:Semantic image segmentation with deep convolutional nets,atrous convolution,and fully connected crfs[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,40(4):834-848.
[11]LIU Z,LIN Y,CAO Y,et al.Swin transformer:Hierarchical vision transformer using shifted windows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:10012-10022.
[12]ZHENG S,LU J,ZHAO H,et al.Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers[C]//Computer Vision and Pattern Recognition.IEEE,2021:6877-6886.
[13]WANG S Y,YANG F.Remote Sensing Image Semantic Seg-mentation Method Based on U-Net Feature Fusion Optimization Strategy[J].Computer Science,2021,48(8):162-168.
[14]LI H,QIU K,CHEN L,et al.SCAttNet:Semantic segmentation network with spatial and channel attention mechanism for high-resolution remote sensing images[J].IEEE Geoscience and Remote Sensing Letters,2020,18(5):905-909.
[15]XU Z,ZHANG W,ZHANG T,et al.HRCNet:high-resolution context extraction network for semantic segmentation of remote sensing images[J].Remote Sensing,2020,13(1):71-93.
[16]YANG X,LI S,CHEN Z,et al.An attention-fused network for semantic segmentation of very-high resolution remote sensing imagery[J].ISPRS Journal of Photogrammetry and Remote Sensing,2021(177):238-262.
[17]WANG Q,GUO L G,CHENG W T.A Method for Extracting Buildings from Remote Sensing Images Based on Lightweight NDFEDet-SOLOv2[J].Journal of Chongqing Technology and Business University(Natural Science Edition),2024(6):20-29.
[18]LIU Y,SHI S,WANG J,et al.Seeing Beyond the Patch:Scale-Adaptive Semantic Segmentation of High-resolution Remote Sensing Imagery based on Reinforcement Learning[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2023:16868-16878.
[19]LIANG Y,YI C X,WANG G Y,et al.Semantic Segmentation of Remote Sensing Images Based on Multi-scale Semantic Encoder-Decoder Network[J].Acta Electronica Sinica,2023,51(11):3199-3214.
[20]LI X,HE H,LI X,et al.Pointflow:Flowing semantics through points for aerial image segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:4217-4226.
[21]YANG X,FAN X,PENG M,et al.Semantic segmentation for remote sensing images based on an AD-HRNet model[J].International Journal of Digital Earth,2022,15(1):2376-2399.
[22]MOU L,HUA Y,ZHU X X.Relation matters:Relational context-aware fully convolutional network for semantic segmentation of high-resolution aerial images[J].IEEE Transactions on Geoscience and Remote Sensing,2020,58(11):7557-7569.
[23]FU J,LIU J,TIAN H,et al.Dual attention network for scene segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:3146-3154.
[24]XIE E,WANG W,YU Z,et al.SegFormer:Simple and efficient design for semantic segmentation with transformers[J].Advances in Neural Information Processing Systems,2021,34:12077-12090.
[25]LI R,ZHENG S,ZHANG C,et al.Multiattention network forsemantic segmentation of fine-resolution remote sensing images[J].IEEE Transactions on Geoscience and Remote Sensing,2020,60:1-13.
[26]SONG Q,LI J,LI C,et al.Fully attentional network for semantic segmentation[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2022:2280-2288.
[27]LI R,WANG L B,ZHANG C,et al.A2-FPN for semantic segmentation of fine-resolution remotely sensed images[J].International Journal of Remote Sensing,2022,43(3):1131-1155.
[28]LI R,ZHENG S,DUAN C,et al.Land cover classification from remote sensing images based on multi-scale fully convolutional network[J].Geo-spatial Information Science,2022,25(2):278-294.
[29]ZHANG Y,YAO T,QIU Z F,et al.Lightweight and Progressively-Scalable Networks for Semantic Segmentation[J].International Journal of Computer Vision,2023,131:2153-2171.
[1] HE Chunhui, GE Bin, ZHANG Chong, XU Hao. Intelligent Error Correction Model for Chinese Idioms Fused with Fixed-length Seq2Seq Network [J]. Computer Science, 2025, 52(5): 227-234.
[2] HAN Daojun, LI Yunsong, ZHANG Juntao, WANG Zemin. Knowledge Graph Completion Method Fusing Entity Descriptions and Topological Structure [J]. Computer Science, 2025, 52(5): 260-269.
[3] LI Xiwang, CAO Peisong, WU Yuying, GUO Shuming, SHE Wei. Study on Security Risk Relation Extraction Based on Multi-view IB [J]. Computer Science, 2025, 52(5): 330-336.
[4] LI Xiaolan, MA Yong. Study on Lightweight Flame Detection Algorithm with Progressive Adaptive Feature Fusion [J]. Computer Science, 2025, 52(4): 64-73.
[5] DENG Ceyu, LI Duantengchuan, HU Yiren, WANG Xiaoguang, LI Zhifei. Joint Inter-word and Inter-sentence Multi-relationship Modeling for Review-basedRecommendation Algorithm [J]. Computer Science, 2025, 52(4): 119-128.
[6] PENG Linna, ZHANG Hongyun, MIAO Duoqian. Complex Organ Segmentation Based on Edge Constraints and Enhanced Swin Unetr [J]. Computer Science, 2025, 52(4): 177-184.
[7] KONG Jialin, ZHANG Qi, WEI Jianze, LI Qi. Adaptive Contextual Learning Method Based on Iris Texture Perception [J]. Computer Science, 2025, 52(4): 185-193.
[8] ZHOU Yi, MAO Kuanmin. Research on Individual Identification of Cattle Based on YOLO-Unet Combined Network [J]. Computer Science, 2025, 52(4): 194-201.
[9] HU Huijuan, QIN Yifeng, XU Heand LI Peng. An Improved YOLOv8 Object Detection Algorithm for UAV Aerial Images [J]. Computer Science, 2025, 52(4): 202-211.
[10] YANG Jincai, YU Moyang, HU Man, XIAO Ming. Automatic Identification and Classification of Topical Discourse Markers [J]. Computer Science, 2025, 52(4): 255-261.
[11] ZHONG Yue, GU Jieming. 3D Reconstruction of Single-view Sketches Based on Attention Mechanism and Contrastive Loss [J]. Computer Science, 2025, 52(3): 77-85.
[12] WANG Mengwei, YANG Zhe. Speaker Verification Method Based on Sub-band Front-end Model and Inverse Feature Fusion [J]. Computer Science, 2025, 52(3): 214-221.
[13] CHEN Guangyuan, WANG Zhaohui, CHENG Ze. Multi-view Stereo Reconstruction with Context-guided Cost Volume and Depth Refinemen [J]. Computer Science, 2025, 52(3): 231-238.
[14] WANG Tao, BAI Xuefei, WANG Wenjian. Selective Feature Fusion for 3D CT Image Segmentation of Renal Cancer Based on Edge Enhancement [J]. Computer Science, 2025, 52(3): 41-49.
[15] WANG Xingbo, ZHANG Hao, GAO Hao, ZHAI Mingliang, XIE Jiucheng. Talking Portrait Synthesis Method Based on Regional Saliency and Spatial Feature Extraction [J]. Computer Science, 2025, 52(3): 58-67.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!