Computer Science ›› 2024, Vol. 51 ›› Issue (3): 174-182.doi: 10.11896/jsjkx.221200032

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Combined Road Segmentation and Contour Extraction for Remote Sensing Images Based on Cascaded U-Net

LI Yu1, YANG Xiangli 1, ZHANG Le 2, LIANG Yalin1, GAO Xian1, YANG Jianxi1   

  1. 1 School of Information Science and Engineering,Chongqing Jiaotong University,Chongqing 400074,China
    2 School of Information and Communication Engineering,University of Electronic Science and Technology of China,Chengdu 611731,China
  • Received:2022-12-05 Revised:2023-04-03 Online:2024-03-15 Published:2024-03-13
  • About author:LI Yu,born in 1997,postgraduate.His main research interest is remote sensing image analysis and application.YANG Jianxi,born in 1977,Ph.D,professor,Ph.D supervisor,is a senior member of CCF(No.68607S).His main research interests include bridge health monitoring and transport infrastructure monitoring.
  • Supported by:
    National Natural Science Foundation of China(62101081) and Science and Technology Research Program of Chongqing Municipal Education Commission(KJZD-M202000702,KJQN202100747).

Abstract: Aiming at the problem that the deep-learning-based model for road information extraction can only output single-task results and the inadequate use of correlation between multiple tasks,a combined road segmentation and contour extraction method based on cascaded U-Net is proposed,which extracts the road contour after fusing the feature map of road semantic segmentation with the original image.Firstly,the U-Net network structure is used to extract the hierarchical features of optical remote sensing images,and the cascaded U-Net structure is introduced to concatenate the features to extract the pixel-level label and contours of roads respectively.Secondly,the attention mechanism module is added to each stage of U-Net to extract spatial context information and deep level features to improve the detection sensitivity of details.Finally,the joint loss function composed of dice coefficient and cross-entropy error is used for the overall training to extract simultaneously the road semantic segmentation and contour results.On the optical remote sensing dataset of the urban area of Ottawa,Canada,the joint extraction method of road information based on cascaded U-Net achieves 42% precision,58% recall,48.2% F1 score and 71.6% mIoU in the segmentation index,and achieves a global optimal threshold(ODS) of 0.896 in the road detection index.The results show that,the model can meet the requirements of joint extraction of road multi-task information and has better detection accuracy.

Key words: Remote sensing image, Road segmentation, Contour extraction, Cascaded U-Net, Attention mechanism

CLC Number: 

  • TP753
[1]LI D R,TONG Q X,LI R X,et al.Current issues in high-resolu-tion Earth observation technology[J].Science China Earth Science,2012,55:1043-1051.
[2]DAI J G,WANG Y,DU Y,et al.Development and prospect of road extraction method for optical remote sensing image[J].Journal of Remote Sensing(Chinese),2020,24(7):804-823.
[3]MENG F,FANG S H.Quasi-automatic extraction of zonal roads from remote sensing images using template matching and BSnake model[J].Geomatics and Information Science of Wuhan University,2012,37(1):39-42.
[4]SCHUBER H,VAN DE GRONDE J J,ROERDINK J B T M.Efficient computation of greyscale path openings [J].Mathematical Morphology Theory and Applications,2016,1:189-202.
[5]LI Q,ZHANG J F,NIU R Q.Damaged road extraction from post-seismic remote sensing images based on GIS and object-oriented method[J].Bulletin of Surveying and Mapping,2015,4:78-81.
[6]ALSHEHHI R,MARPU P R,WOON W L,et al.Simultaneous extraction of roads and buildings in remote sensing imagery with convolutional neural networks [J].ISPRS Journal of Photogrammetry and Remote Sensing,2017,130:139-149.
[7]LIU X,WANG G H,YANG H,et al.Road extraction from remote sensing image based on full convolutional network [J].Remote Sensing Information,2018,33(1):69-75.
[8]ZHANG Y H,XIA G H,KAN X,et al.Road extraction from multi-source high resolution remote sensing image based on fullyconvolution network[J].Journal of Computer Applications,2018,38(7):2070-2075.
[9]YUAN M,LIU Z,WANG F.Using the wide-range attentionU-Net for road segmentation [J].Remote Sensing Letters,2019,10(5):506-515.
[10]LAN M,ZHANG Y,ZHANG L,et al.Global context based automatic road segmentation via dilated convolutional neural network [J].Information Sciences,2020,535:156-171.
[11]CUI L,ZHANG P,CHE J.Overview of Deep Neural Network Based Classification Algorithms for Remote Sensing Image[J].Computer Science,2018,45(S1):50-53.
[12]LUO L,WANG J X,CHEN S B,et al.BDTNet:Road extraction by Bi-Direction transformer from remote sensing images[J].IEEE Geoscience and Remote Sensing Letters,2022,19:1-5.
[13]LI J,MENG Y,DORJEE D,et al.Automatic road extractionfrom remote sensing imagery using ensemble learning and postprocessing[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2021,14:10535-10547.
[14]YIN W,QIAN M,WANG L,et al.Road extraction from satellite images with iterative cross-task feature enhancement[J].Neurocomputing,2022,506:300-310.
[15]ZHU X,TUIA D,MOU L,et al.Deep learning in remote sen-sing [J].IEEE Geoscience and Remote Sensing Magazine,2017,4(4):8-36.
[16]TONG X Y,XIA G S,ZHONG Y,et al.Exploiting deep features for remote sensing image retrieval:a systematic investigation [J].IEEE Transaction on Big Data,2019,6(3):507-521.
[17]MNIH V,HINTON G E.Learning to detect roads in high-resolution aerial images[C]// European Conference on Computer Vision.Berlin,Heidelberg:Springer,2010:210-223.
[18]WANG Y,PENG Y,LI W,et al.DDU-Net:dual-decoder-U-Net for road extraction using high-resolution remote sensing images[J].IEEE Transactions on Geoscience and Remote Sensing,2022,60:1-12.
[19]ZHANG Z X,LIU Q J,WANG Y H.Road extraction by deep residual U-Net[J].IEEE Geoscience and Remote Sensing Letters,2018,15(5):749-753.
[20]XIN J,ZHANG X C,ZHANG Z Q,et al.Road extraction ofhigh resolution remote sensing images derived from Dense U-Net[J].Remote Sensing,2019,21(11):2499.
[21]TAN X W,XIAO Z F,WAN Q,et al.Scale sensitive neural network for road segmentation in high-resolution remote sensing images [J].IEEE Geoscience and Remote Sensing Letters,2021,18(3):533-537.
[22]LI X,ZHANG S,PAN X,et al.Straight road edge detection from high-resolution remote sensing images based on the ridgelet transform with the revised parallel-beam Radon transform[J].International Journal of Remote Sensing,2010,31(19):5041-5059.
[23]HUANG X,ZHANG L P.Road centreline extraction from high-resolution imagery based on multiscale structural features and support vector machines [J].International Journal of Remote Sensing,2009,30(8):1977-1987.
[24]XU Z H,SUN Y X,LIU M.iCurb:imitation learning-based detection of road curbs using aerial images for autonomous driving [J].IEEE Robotics and Automation Letters,2021,6(2):1097-1104.
[25]SIRONI A,TURETKEN E,LEPETIT V,et al.Multiscale centerline detection [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(7):1327-1341.
[26]GUO Q,WANG Z P.A self-supervised learning framework for road centerline extraction from high-resolution remote sensing images [J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2020,13:4451-446.
[27]ZHANG Y,YANG Q.A Survey on Multi-Task Learning [J].National Science Review,2018,5(1):30-43.
[28]HEIDLER K,MOU L C,BAUMHOER C.HED-UNet:com-bined segmentation and edge detection for monitoring the antarctic coastline [J].IEEE Transactions on Geoscience and Remote Sensing,2021,60:4300514.
[29]CHENG G L,WANG Y,XU S B,et al.Automatic road detection and centerline extraction via cascaded end-to-end convolutional neural network [J].IEEE Transactions on Geoscience and Remote Sensing,2017,55(6):3322-3337.
[30]SHAO Z F,ZHOU Z F,HUANG X,et al.MRENet:Simultaneous extraction of road surface and road centerline in complex urban scenes from very high-resolution images[J].Remote Sen-sing,2021,13(2):239.
[31]LIU Y H,YAO J,LU X H,et al.RoadNet:learning to comprehensively analyze road networks in complex urban scenes from high-resolution remotely sensed images [J].IEEE Transactions on Geoscience and Remote Sensing,2018,57(4):2043-2056.
[32]LU X Y,ZHONG Y F,ZHENG Z,et al.Cascaded multi-task road extraction network for road surface,centerline,and edge extraction [J].IEEE Transactions on Geoscience and Remote Sensing,2022,60:5621414.
[33]GHANDORH H,BOULILA W,MASOOS S,et al.Semanticsegmentation and edge detection-approach to road detection in very high resolution satellite images[J].Remote Sensing,2022,14(3):613.
[34]RONNEBERGER O,FISCHER P,BROX T.U-net:Convolu-tional networks for biomedical image segmentation[C]// International Conference on Medical Image Computing and Computer Assisted Intervention.Munich,Germany,2015:234-241.
[35]WOO S,PARK J,LEE J Y,et al.CBAM:Convolutional block attention module[C]// Proceedings of the European Conference on Computer Vision.Munich,Germany,2018:3-19.
[36]SHAO S,XIAO L,LIN L,et al.Road extraction convolutional neural network with embedded attention mechanism for remote sensing imagery[J].Remote Sensing,2022,14(9):2061.
[37]EVERINGHAM M,ALI ESLAMI S M,VAN GOOL L,et al.The PASCAL visual object classes challenge:a retrospective [J].International Journal of Computer Vision,2015,111(1):98-136.
[38]LIU Y,CHENG M M,HU X W,et al.Richer convolutional features for edge detection [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2019,41(8):1939-1946.
[39]BADRINARAYANAN V,KENDALL A,CIPOLLA R.Seg-Net:A deep convolutional encoder-decoder architecture for image segmentation [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(12):2481-2495.
[1] ZHANG Mingdao, ZHOU Xin, WU Xiaohong, QING Linbo, HE Xiaohai. Unified Fake News Detection Based on Semantic Expansion and HDGCN [J]. Computer Science, 2024, 51(4): 299-306.
[2] WANG Ruiping, WU Shihong, ZHANG Meihang, WANG Xiaoping. Review of Vision-based Neural Network 3D Dynamic Gesture Recognition Methods [J]. Computer Science, 2024, 51(4): 193-208.
[3] WANG Zihong, SHAO Yingxia, HE Jiyuan, LIU Jinbao. Sequential Recommendation Based on Multi-space Attribute Information Fusion [J]. Computer Science, 2024, 51(3): 102-108.
[4] HAO Ran, WANG Hongjun, LI Tianrui. Deep Neural Network Model for Transmission Line Defect Detection Based on Dual-branch Sequential Mixed Attention [J]. Computer Science, 2024, 51(3): 135-140.
[5] ZHANG Yang, XIA Ying. Object Detection Method with Multi-scale Feature Fusion for Remote Sensing Images [J]. Computer Science, 2024, 51(3): 165-173.
[6] LIAO Meng, JIA Zhen, LI Tianrui. Chinese Named Entity Recognition Based on Label Information Fusion and Multi-task Learning [J]. Computer Science, 2024, 51(3): 198-204.
[7] SUN Shounan, WANG Jingbin, WU Renfei, YOU Changkai, KE Xifan, HUANG Hao. TMGAT:Graph Attention Network with Type Matching Constraint [J]. Computer Science, 2024, 51(3): 235-243.
[8] LIU Xuheng, BAI Zhengyao, XU Zhu, DU Jiajin, XIAO Xiao. Multi-guided Point Cloud Registration Network Combined with Attention Mechanism [J]. Computer Science, 2024, 51(2): 142-150.
[9] ZHANG Guodong, CHEN Zhihua, SHENG Bin. Infrared Small Target Detection Based on Dilated Convolutional Conditional GenerativeAdversarial Networks [J]. Computer Science, 2024, 51(2): 151-160.
[10] ZHANG Feng, HUANG Shixin, HUA Qiang, DONG Chunru. Novel Image Classification Model Based on Depth-wise Convolution Neural Network andVisual Transformer [J]. Computer Science, 2024, 51(2): 196-204.
[11] LIU Changxin, WU Ning, HU Lirui, GAO Ba, GAO Xueshan. Recursive Gated Convolution Based Super-resolution Network for Remote Sensing Images [J]. Computer Science, 2024, 51(2): 205-216.
[12] LIAO Xingbin, QIAN Yangge, WANG Qianlei, QIN Xiaolin. Hierarchical Document Classification Method Based on Improved Self-attention Mechanism and Representation Learning [J]. Computer Science, 2024, 51(2): 238-244.
[13] WANG Weijia, XIONG Wenzhuo, ZHU Shengjie, SONG Ce, SUN He, SONG Yulong. Method of Infrared Small Target Detection Based on Multi-depth Feature Connection [J]. Computer Science, 2024, 51(1): 175-183.
[14] SHI Dianxi, LIU Yangyang, SONG Linna, TAN Jiefu, ZHOU Chenlei, ZHANG Yi. FeaEM:Feature Enhancement-based Method for Weakly Supervised Salient Object Detection via Multiple Pseudo Labels [J]. Computer Science, 2024, 51(1): 233-242.
[15] YI Liu, GENG Xinyu, BAI Jing. Hierarchical Multi-label Text Classification Algorithm Based on Parallel Convolutional Network Information Fusion [J]. Computer Science, 2023, 50(9): 278-286.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!