Computer Science ›› 2024, Vol. 51 ›› Issue (11): 148-156.doi: 10.11896/jsjkx.231000148

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Study on Road Crack Detection Based on Weakly Supervised Semantic Segmentation

ZHAO Weidong, LU Ming, ZHANG Rui   

  1. School of Software,Fudan University,Shanghai 200433,China
    Shanghai Key Laboratory of Data Science,Shanghai 200433,China
  • Received:2023-10-23 Revised:2024-03-07 Online:2024-11-15 Published:2024-11-06
  • About author:ZHAO Weidong,born in 1971,Ph.D,associate professor.His main research interests include machine learning,deep learning and recommender systems.
  • Supported by:
    National Natural Science Foundation of China(71971066).

Abstract: Most of the existing weakly supervised semantic segmentation methods are based on the process of blocking before detection,which increases the annotation workload.However,the existing automatic block classification methods input all blocks into the model to predict the block category,increasing the number of blocks that are misjudged and affecting the performance of subsequent semantic segmentation.Aiming at the above problems,this paper proposes a road crack block classification model based on deep reinforcement learning.According to characteristics of road crack images,the states,actions,and rewards obtained by the agents are designed.The agent is trained to select crack blocks independently,and the selection results are used as block labels for multi-size block road crack detection.Through comparative experiments on several datasets,it is proved that the propsoed model outperforms existing methods in terms of road crack segmentation performance and crack width measurement accuracy.

Key words: Road crack detection, Weakly supervision, Semantic segmentation, Crack blocks, Deep reinforcement learning

CLC Number: 

  • TP391
[1] LI B,WANG K C P,ZHANG A,et al.Automatic classification of pavement crack using deep convolutional neural network[J].International Journal of Pavement Engineering,2020,21(4):457-463.
[2] PING P,YANG X,GAO Z.A Deep Learning Approach forStreet Pothole Detection[C]//2020 IEEE Sixth International Conference on Big Data Computing Service and Applications,Oxford,United Kingdom.IEEE,2020:199-205.
[3] HONG Y,LEE S,YOO S B.AugMoCrack:Augmented morphological attention network for weakly supervised crack detection[J].Electronics Letters,2022,58(17):651-653.
[4] ZHANG Y,LI Q,ZHAO X,et al.TB-Net:A Three-StreamBoundary-Aware Network for Fine-Grained Pavement Disease Segmentation[C]//2021 IEEE Winter Conference on Applications of Computer Vision,Waikoloa.IEEE,2021:3654-3663.
[5] YANG F,ZHANG L,YU S,et al.Feature Pyramid and Hierarchical Boosting Network for Pavement Crack Detection[J].IEEE Transactions on Intelligent Transportation Systems,2020,21(4):1525-1535.
[6] ZOU Q,ZHANG Z,LI Q,et al.DeepCrack:Learning Hierarchical Convolutional Features for Crack Detection[J].IEEE Transactions on Image Processing,2019,28:1498-1512.
[7] LAU S L H,CHONG E K P,YANG X,et al.Automated Pavement Crack Segmentation Using U-Net-Based Convolutional Neural Network[J].IEEE Access,2020,8:114892-114899.
[8] TANG W,HUANG S,ZHAO Q,et al.An Iteratively Optimized Patch Label Inference Network for Automatic Pavement Distress Detection[J].IEEE Transactions on Intelligent Transportation Systems,2022,23(7):8652-8661.
[9] YAN K,ZHANG Z.Automated Asphalt Highway PavementCrack Detection Based on Deformable Single Shot Multi-Box Detector Under a Complex Environment[J].IEEE Access,2021,9:150925-150938.
[10] ZHU J S,SONG J B.Weakly supervised network based intelligent identification of cracks in asphalt concrete bridge deck-ScienceDirect[J].Alexandria Engineering Journal,2020,59(3):1307-1317.
[11] HUANG G,HUANG S,HUANGFU L,et al.Weakly Super-vised Patch Label Inference Network with Image Pyramid for Pavement Diseases Recognition in the Wild[C]//ICASSP 2021-2021 IEEE International Conference on Acoustics,Speech and Signal Processing.Toronto,2021:7978-7982.
[12] XU B,LIU C.Pavement crack detection algorithm based on ge-nerative adversarial network and convolutional neural network under small samples[J].Measurement,2022,196(9):111219.
[13] LI Y,BAO T,XU B,et al.A deep residual neural networkframework with transfer learning for concretedams patch-level crack classification and weakly-supervised localization[J].Measurement,2022,188(1):110641.
[14] WANG H,LI Y,DANG L M,et al.Pixel-level tunnel crack segmentation using a weakly supervised annotation approach[J].Computers in Industry,2021,133(12):103545.
[15] DONG Z,WANG J,CUI B,et al.Patch-based weakly supervised semantic segmentation network for crack detection[J].Construction and Building Materials,2020,258(10):1-14.
[16] RILL-GARCÍA R,DOKLADALOVA E,DOKLÁDAL P.Pixel-accurate road crack detection in presence of inaccurate annotations[J].Neurocomputing,2022,480(4):1-13.
[17] INOUE Y,NAGAYOSHI H.Crack Detection as a Weakly-Supervised Problem:TowardsAchieving Less Annotation-Intensive Crack Detectors[C]//2020 25th International Conference on Pattern Recognition,Milan,Italy.2021:65-72.
[18] CAICEDO J C,LAZEBNIK S.Active Object Localization with Deep Reinforcement Learning[C]//2015 IEEE International Conference on Computer Vision.Santiago,Chile,2015:2488-2496.
[19] SILVER D,HUBERT T,SCHRITTWIESER J,et al.Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm[EB/OL].https://arxiv.org/abs/1712.01815.
[20] YE D,LIU Z,SUN M,et al.Mastering Complex Control in MOBA Games with Deep Reinforcement Learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:6672-6679.
[21] AHN J,KWAK S.Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City,2018:4981-4990.
[22] HUANG Z,WANG X,WANG J,et al.Weakly-Supervised Semantic Segmentation Network with Deep Seeded Region Gro-wing[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City,2018:7014-7023.
[23] WANG X,YOU S,LI X,et al.Weakly-Supervised SemanticSegmentation by Iteratively Mining Common Object Features[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City,2018:1354-1362.
[21] SUN G,WANG W,DAI J,et al.Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentation[C]//Computer Vision-ECCV 2020.2020:347-365.
[22] WANG Y,ZHANG J,KAN M,et al.Self-Supervised Equiva-riant Attention Mechanism for Weakly Supervised Semantic Segmentation[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle,2020:12272-12281.
[23] YANG X,LI H,YU Y,et al.Automatic Pixel-Level Crack Detection and Measurement Using Fully Convolutional Network[J].Computer-Aided Civil and Infrastructure Engineering,2018,33(12):1090-1109.
[24] SELVARAJU R R,COGSWELL M,DAS A,et al.Grad-CAM:Visual Explanations from Deep Networks via Gradient-based Localization[C]//2017 IEEE International Conference on Computer Vision.Venice,2017:618-626.
[1] WANG Tianjiu, LIU Quan, WU Lan. Offline Reinforcement Learning Algorithm for Conservative Q-learning Based on Uncertainty Weight [J]. Computer Science, 2024, 51(9): 265-272.
[2] ZHOU Wenhui, PENG Qinghua, XIE Lei. Study on Adaptive Cloud-Edge Collaborative Scheduling Methods for Multi-object State Perception [J]. Computer Science, 2024, 51(9): 319-330.
[3] WANG Qian, HE Lang, WANG Zhanqing, HUANG Kun. Road Extraction Algorithm for Remote Sensing Images Based on Improved DeepLabv3+ [J]. Computer Science, 2024, 51(8): 168-175.
[4] GAO Yuzhao, NIE Yiming. Survey of Multi-agent Deep Reinforcement Learning Based on Value Function Factorization [J]. Computer Science, 2024, 51(6A): 230300170-9.
[5] HUANG Haixin, CAI Mingqi, WANG Yuyao. Review of Point Cloud Semantic Segmentation Based on Graph Convolutional Neural Networks [J]. Computer Science, 2024, 51(6A): 230400196-7.
[6] WANG Shuanqi, ZHAO Jianxin, LIU Chi, WU Wei, LIU Zhao. Fuzz Testing Method of Binary Code Based on Deep Reinforcement Learning [J]. Computer Science, 2024, 51(6A): 230800078-7.
[7] SUN Jifei, JIA Kebin. Classification and Detection Algorithm of Ground-based Cloud Images Based on Multi-scale Features [J]. Computer Science, 2024, 51(6A): 230400041-6.
[8] WANG Guogang, DONG Zhihao. Lightweight Image Semantic Segmentation Based on Attention Mechanism and Densely AdjacentPrediction [J]. Computer Science, 2024, 51(6A): 230300204-8.
[9] WU Yibo, HAO Yingguang, WANG Hongyu. Rice Defect Segmentation Based on Dual-stream Convolutional Neural Networks [J]. Computer Science, 2024, 51(6A): 230600107-8.
[10] ZHANG Xiaoqing, WANG Qingwang, QU Xin, SHEN Shiquan, WU Changyi, LIU Ju. Direction-aware Pyramidal Aggregation Network for Road Centerline Extraction [J]. Computer Science, 2024, 51(6A): 230400101-7.
[11] XIAO Yahui, ZHANG Zili, HU Xinrong, PENG Tao, ZHANG Jun. Clothing Image Segmentation Method Based on Deeplabv3+ Fused with Attention Mechanism [J]. Computer Science, 2024, 51(6A): 230900153-7.
[12] LI Danyang, WU Liangji, LIU Hui, JIANG Jingqing. Deep Reinforcement Learning Based Thermal Awareness Energy Consumption OptimizationMethod for Data Centers [J]. Computer Science, 2024, 51(6A): 230500109-8.
[13] YANG Xiuwen, CUI Yunhe, QIAN Qing, GUO Chun, SHEN Guowei. COURIER:Edge Computing Task Scheduling and Offloading Method Based on Non-preemptivePriorities Queuing and Prioritized Experience Replay DRL [J]. Computer Science, 2024, 51(5): 293-305.
[14] HUANG Wenke, TENG Fei, WANG Zidan, FENG Li. Image Segmentation Based on Deep Learning:A Survey [J]. Computer Science, 2024, 51(2): 107-116.
[15] LI Junwei, LIU Quan, XU Yapeng. Option-Critic Algorithm Based on Mutual Information Optimization [J]. Computer Science, 2024, 51(2): 252-258.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!