计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (3): 174-182.doi: 10.11896/jsjkx.221200032

• 计算机图形学&多媒体 • 上一篇    下一篇

基于级联U-Net的遥感影像道路分割和轮廓提取方法

李余1, 杨祥立1, 张乐2, 梁雅麟1, 高显1, 杨建喜1   

  1. 1 重庆交通大学信息科学与工程学院 重庆400074
    2 电子科技大学信息与通信工程学院 成都611731
  • 收稿日期:2022-12-05 修回日期:2023-04-03 出版日期:2024-03-15 发布日期:2024-03-13
  • 通讯作者: 杨建喜(yjx@cqjtu.edu.cn)
  • 作者简介:(leeeeyu@163.com)
  • 基金资助:
    国家自然科学基金(62101081);重庆市教委科学技术研究项目(KJZD-M202000702,KJQN202100747)

Combined Road Segmentation and Contour Extraction for Remote Sensing Images Based on Cascaded U-Net

LI Yu1, YANG Xiangli 1, ZHANG Le 2, LIANG Yalin1, GAO Xian1, YANG Jianxi1   

  1. 1 School of Information Science and Engineering,Chongqing Jiaotong University,Chongqing 400074,China
    2 School of Information and Communication Engineering,University of Electronic Science and Technology of China,Chengdu 611731,China
  • Received:2022-12-05 Revised:2023-04-03 Online:2024-03-15 Published:2024-03-13
  • About author:LI Yu,born in 1997,postgraduate.His main research interest is remote sensing image analysis and application.YANG Jianxi,born in 1977,Ph.D,professor,Ph.D supervisor,is a senior member of CCF(No.68607S).His main research interests include bridge health monitoring and transport infrastructure monitoring.
  • Supported by:
    National Natural Science Foundation of China(62101081) and Science and Technology Research Program of Chongqing Municipal Education Commission(KJZD-M202000702,KJQN202100747).

PDF (PC)

126

摘要： 针对基于深度学习的遥感图像道路信息提取模型往往只能输出单任务结果且多任务之间相关性利用不充分的问题,提出了一种基于级联U-Net的道路语义分割和轮廓联合检测方法,将道路语义分割后的特征图与原始图像融合后进行道路轮廓的提取,实现道路语义分割和边界轮廓的联合训练。首先使用U-Net网络结构提取光学遥感图像丰富的层次化特征,通过级联结构将特征串联融合,分别用于提取道路的语义类别和边界轮廓。其次在每级U-Net结构中引入注意力机制模块,进行空间上下文信息和深层次特征提取,改善网络提取过程中出现的细节模糊现象。最后,使用骰子系数和交叉熵误差组成的联合损失函数进行多任务整体训练,实现深度学习模型对遥感图像中道路语义类别和边界轮廓的同时提取。通过在加拿大渥太华城市地区的光学遥感数据集上进行实验,基于级联U-Net的道路信息联合提取方法在分割指标上分别获得了42%的精确度、58%的召回率、48.2%的F1分数以及71.6%的平均交并比,在道路检测指标上取得了0.896的全局最佳阈值(ODS)。结果表明,该模型在满足联合提取道路多任务信息的同时具有更优的检测精度。

关键词: 遥感影像, 道路分割, 轮廓提取, 级联U-Net, 注意力机制

Abstract: Aiming at the problem that the deep-learning-based model for road information extraction can only output single-task results and the inadequate use of correlation between multiple tasks,a combined road segmentation and contour extraction method based on cascaded U-Net is proposed,which extracts the road contour after fusing the feature map of road semantic segmentation with the original image.Firstly,the U-Net network structure is used to extract the hierarchical features of optical remote sensing images,and the cascaded U-Net structure is introduced to concatenate the features to extract the pixel-level label and contours of roads respectively.Secondly,the attention mechanism module is added to each stage of U-Net to extract spatial context information and deep level features to improve the detection sensitivity of details.Finally,the joint loss function composed of dice coefficient and cross-entropy error is used for the overall training to extract simultaneously the road semantic segmentation and contour results.On the optical remote sensing dataset of the urban area of Ottawa,Canada,the joint extraction method of road information based on cascaded U-Net achieves 42% precision,58% recall,48.2% F1 score and 71.6% mIoU in the segmentation index,and achieves a global optimal threshold(ODS) of 0.896 in the road detection index.The results show that,the model can meet the requirements of joint extraction of road multi-task information and has better detection accuracy.

Key words: Remote sensing image, Road segmentation, Contour extraction, Cascaded U-Net, Attention mechanism

中图分类号: 

  • TP753
[1]LI D R,TONG Q X,LI R X,et al.Current issues in high-resolu-tion Earth observation technology[J].Science China Earth Science,2012,55:1043-1051.
[2]DAI J G,WANG Y,DU Y,et al.Development and prospect of road extraction method for optical remote sensing image[J].Journal of Remote Sensing(Chinese),2020,24(7):804-823.
[3]MENG F,FANG S H.Quasi-automatic extraction of zonal roads from remote sensing images using template matching and BSnake model[J].Geomatics and Information Science of Wuhan University,2012,37(1):39-42.
[4]SCHUBER H,VAN DE GRONDE J J,ROERDINK J B T M.Efficient computation of greyscale path openings [J].Mathematical Morphology Theory and Applications,2016,1:189-202.
[5]LI Q,ZHANG J F,NIU R Q.Damaged road extraction from post-seismic remote sensing images based on GIS and object-oriented method[J].Bulletin of Surveying and Mapping,2015,4:78-81.
[6]ALSHEHHI R,MARPU P R,WOON W L,et al.Simultaneous extraction of roads and buildings in remote sensing imagery with convolutional neural networks [J].ISPRS Journal of Photogrammetry and Remote Sensing,2017,130:139-149.
[7]LIU X,WANG G H,YANG H,et al.Road extraction from remote sensing image based on full convolutional network [J].Remote Sensing Information,2018,33(1):69-75.
[8]ZHANG Y H,XIA G H,KAN X,et al.Road extraction from multi-source high resolution remote sensing image based on fullyconvolution network[J].Journal of Computer Applications,2018,38(7):2070-2075.
[9]YUAN M,LIU Z,WANG F.Using the wide-range attentionU-Net for road segmentation [J].Remote Sensing Letters,2019,10(5):506-515.
[10]LAN M,ZHANG Y,ZHANG L,et al.Global context based automatic road segmentation via dilated convolutional neural network [J].Information Sciences,2020,535:156-171.
[11]CUI L,ZHANG P,CHE J.Overview of Deep Neural Network Based Classification Algorithms for Remote Sensing Image[J].Computer Science,2018,45(S1):50-53.
[12]LUO L,WANG J X,CHEN S B,et al.BDTNet:Road extraction by Bi-Direction transformer from remote sensing images[J].IEEE Geoscience and Remote Sensing Letters,2022,19:1-5.
[13]LI J,MENG Y,DORJEE D,et al.Automatic road extractionfrom remote sensing imagery using ensemble learning and postprocessing[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2021,14:10535-10547.
[14]YIN W,QIAN M,WANG L,et al.Road extraction from satellite images with iterative cross-task feature enhancement[J].Neurocomputing,2022,506:300-310.
[15]ZHU X,TUIA D,MOU L,et al.Deep learning in remote sen-sing [J].IEEE Geoscience and Remote Sensing Magazine,2017,4(4):8-36.
[16]TONG X Y,XIA G S,ZHONG Y,et al.Exploiting deep features for remote sensing image retrieval:a systematic investigation [J].IEEE Transaction on Big Data,2019,6(3):507-521.
[17]MNIH V,HINTON G E.Learning to detect roads in high-resolution aerial images[C]// European Conference on Computer Vision.Berlin,Heidelberg:Springer,2010:210-223.
[18]WANG Y,PENG Y,LI W,et al.DDU-Net:dual-decoder-U-Net for road extraction using high-resolution remote sensing images[J].IEEE Transactions on Geoscience and Remote Sensing,2022,60:1-12.
[19]ZHANG Z X,LIU Q J,WANG Y H.Road extraction by deep residual U-Net[J].IEEE Geoscience and Remote Sensing Letters,2018,15(5):749-753.
[20]XIN J,ZHANG X C,ZHANG Z Q,et al.Road extraction ofhigh resolution remote sensing images derived from Dense U-Net[J].Remote Sensing,2019,21(11):2499.
[21]TAN X W,XIAO Z F,WAN Q,et al.Scale sensitive neural network for road segmentation in high-resolution remote sensing images [J].IEEE Geoscience and Remote Sensing Letters,2021,18(3):533-537.
[22]LI X,ZHANG S,PAN X,et al.Straight road edge detection from high-resolution remote sensing images based on the ridgelet transform with the revised parallel-beam Radon transform[J].International Journal of Remote Sensing,2010,31(19):5041-5059.
[23]HUANG X,ZHANG L P.Road centreline extraction from high-resolution imagery based on multiscale structural features and support vector machines [J].International Journal of Remote Sensing,2009,30(8):1977-1987.
[24]XU Z H,SUN Y X,LIU M.iCurb:imitation learning-based detection of road curbs using aerial images for autonomous driving [J].IEEE Robotics and Automation Letters,2021,6(2):1097-1104.
[25]SIRONI A,TURETKEN E,LEPETIT V,et al.Multiscale centerline detection [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(7):1327-1341.
[26]GUO Q,WANG Z P.A self-supervised learning framework for road centerline extraction from high-resolution remote sensing images [J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2020,13:4451-446.
[27]ZHANG Y,YANG Q.A Survey on Multi-Task Learning [J].National Science Review,2018,5(1):30-43.
[28]HEIDLER K,MOU L C,BAUMHOER C.HED-UNet:com-bined segmentation and edge detection for monitoring the antarctic coastline [J].IEEE Transactions on Geoscience and Remote Sensing,2021,60:4300514.
[29]CHENG G L,WANG Y,XU S B,et al.Automatic road detection and centerline extraction via cascaded end-to-end convolutional neural network [J].IEEE Transactions on Geoscience and Remote Sensing,2017,55(6):3322-3337.
[30]SHAO Z F,ZHOU Z F,HUANG X,et al.MRENet:Simultaneous extraction of road surface and road centerline in complex urban scenes from very high-resolution images[J].Remote Sen-sing,2021,13(2):239.
[31]LIU Y H,YAO J,LU X H,et al.RoadNet:learning to comprehensively analyze road networks in complex urban scenes from high-resolution remotely sensed images [J].IEEE Transactions on Geoscience and Remote Sensing,2018,57(4):2043-2056.
[32]LU X Y,ZHONG Y F,ZHENG Z,et al.Cascaded multi-task road extraction network for road surface,centerline,and edge extraction [J].IEEE Transactions on Geoscience and Remote Sensing,2022,60:5621414.
[33]GHANDORH H,BOULILA W,MASOOS S,et al.Semanticsegmentation and edge detection-approach to road detection in very high resolution satellite images[J].Remote Sensing,2022,14(3):613.
[34]RONNEBERGER O,FISCHER P,BROX T.U-net:Convolu-tional networks for biomedical image segmentation[C]// International Conference on Medical Image Computing and Computer Assisted Intervention.Munich,Germany,2015:234-241.
[35]WOO S,PARK J,LEE J Y,et al.CBAM:Convolutional block attention module[C]// Proceedings of the European Conference on Computer Vision.Munich,Germany,2018:3-19.
[36]SHAO S,XIAO L,LIN L,et al.Road extraction convolutional neural network with embedded attention mechanism for remote sensing imagery[J].Remote Sensing,2022,14(9):2061.
[37]EVERINGHAM M,ALI ESLAMI S M,VAN GOOL L,et al.The PASCAL visual object classes challenge:a retrospective [J].International Journal of Computer Vision,2015,111(1):98-136.
[38]LIU Y,CHENG M M,HU X W,et al.Richer convolutional features for edge detection [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2019,41(8):1939-1946.
[39]BADRINARAYANAN V,KENDALL A,CIPOLLA R.Seg-Net:A deep convolutional encoder-decoder architecture for image segmentation [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(12):2481-2495.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发