Computer Science ›› 2022, Vol. 49 ›› Issue (7): 120-126.doi: 10.11896/jsjkx.210500157

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction

CHENG Cheng, JIANG Ai-lian   

  1. College of Information and Computer,Taiyuan University of Technology,Jinzhong,Shanxi 030600,China
  • Received:2021-05-24 Revised:2021-09-07 Online:2022-07-15 Published:2022-07-12
  • About author:CHENG Cheng,born in 1996,postgra-duate,is a member of China Computer Federation.His main research interests include deep learning and semantic segmentation.
    JIANG Ai-lian,born in 1969,Ph.D,associate professor,is a member of China Computer Federation.Her main research interests include big data analysis and processing,feature selection,artificial intelligence and computer vision.
  • Supported by:
    Scientific Research Funding Project for Returned Overseas Scholars in Shanxi Province(2017-051).

Abstract: The application of deep learning in the field of image semantic segmentation has greatly improved the accuracy of segmentation,but due to the limitations of speed and memory,these models can not be directly applied to embedded devices for real-time segmentation.Aiming at the problems of complex network structure and huge computation cost of semantic segmentation model,a real-time semantic segmentation algorithm based on multi-path feature extraction combined with edge detection algorithm is proposed.The model uses Sobel operator,Scharr operator and Laplacian operator to extract the contour information of the image.The algorithm designs the spatial path to extract the spatial position information of the image,designs the semantic path to extract the advanced semantic information of the image,and uses the edge detection path to extract the representative texture features of the image.The ghost lightweight module is used to reduce the amount of model parameters and improve the segmentation speed of the algorithm.Experimental results on 480 pixel and 360 pixel CamVid dataset show that the segmentation accuracy of the model can be improved on the three edge detection operators,especially when the Sobel operator with the size of 3×3 is added,the performance of the algorithm is improved most obviously,and the segmentation accuracy reaches 42.9% on the basis of the image processing speed of 349 frames/s on CamVid test set.Both the segmentation accuracy and segmentation speed achieve good results,and achieve a good balance between real-time and accuracy.

Key words: Deep learning, Edge detection, Feature fusion, Multi-feature extraction, Semantic segmentation in real-time

CLC Number: 

  • TP391
[1]CHEN Q S,TAO Y,SHEN F H,et al.Semantic segmentation of images based on contextual structure[J].Journal of Chongqing University of Posts and Telecommunications(Natural Science Edition),2020,32(2):287-294.
[2]CHAO Y,XIAO N F.Research on Robotic Grasping Methods Based on Semantic Segmentation and Brain Computer Interface[J].Journal of Chongqing University of Technology(Natural Science),2020,34(3):128 -136,151.
[3]LECUN Y,BOTTOU L.Gradient-based learning applied to docu-ment recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[4]LONG J,SHEKHAMER E,DARRELL T.Fully Convolutional Networks for Semantic Segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,39(4):640-651.
[5]RONNEBERGER O,FISCHER P,BROX T,et al.U-net:Con-volutional networks for biomedical image segmentation[J].Medical Image Computing and Computer Assisted Intervention,2015,28(4):234-241.
[6]ZHAO H,SHI J,QI X,et al.Pyramid scene parsing network[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Hawaii:IEEE Press,2017:2881-2890.
[7]CHEN L C,PAPANDREOU G,KOKKINOS I,et al.DeepLab:Semantic Image Segmentation with Deep Convolutional Nets,Atrous Convolution,and Fully Connected CRFs[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2016,40(4):834-848.
[8]PASZKE A,CHAURASIA A,KIM A.ENet:a deep neural network architecture for real-time semantic segmentation[J].ar-Xiv:1606.02147,2016.
[9]YU C,WANG J,PENG C,et al.Bisenet:Bilateral segmentation network for real-time semantic segmentation[C]//Proceedings of the European Conference on Computer Vision.Munich,Germany,2018:325-341.
[10]LI H,XIONG P,FAN H,et al.Dfanet:Deep feature aggregation for real-time semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Seoul:IEEE Press,2019:9522-9531.
[11]HAN K,WANG Y,TIAN Q,et al.GhostNet:More Features From Cheap Operations[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Seattle:IEEE Press,2020:1577-1586.
[12]LIN G S,MILAN A,SHEN C H,et al.Refinenet:multi-path refinement networks for high-resolution semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:5168-5177.
[13]SZEGEDY C,VANHOUCKE V,IOFFE S,et al.Rethinking the inception architecture for computer vision[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas,NV,2016:2818-2826.
[14]CHOLLET F.Xception:deep learning with depth wise separable convolutions[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition.Honolulu,2017:1800-1807.
[15]SANDLER M,HOWARD A,ZHU M,et al.MobileNetV2:inverted residuals and linear bottlenecks.conference [C]//IEEE Conference on Computer Vision and Pattern Recognition.Salt Lake City:IEEE,2018:4510-4520.
[16]ZHUANG L,LI J,SHEN Z,et al.Learning Efficient Convolutional Networks through Network Slimming [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition.Honolulu,2017:2755-2763.
[17]ZHAO H,QI X,SHEN X,et al.Icnet for real-time semantic segmentation on high-resolution images[C]//Proceedings of the European Conference on Computer Vision(ECCV).Munich,Germany,2018:405-420.
[18]MA N,ZHANG X,ZHENG H T,et al.Shufflenet v2:Practical guidelines for efficient CNN architecture design[C]//Procee-dings of the European Conference on Computer Vision.Munich,Germany,2018:116-131.
[19]WANG Y,ZHOU Q,LIU J,et al.LEDNet:A Lightweight Encoder-Decoder Network for Real-Time Semantic Segmentation[C]//2019 IEEE International Conference on Image Processing(ICIP).Taipei:IEEE Press,2019:126-172.
[20]BADRINARAYANAN V,KENDALL A,CIPOLLA R.Segnet:A deep convolutional encoder-decoder architecture for image segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(12):2481-2495.
[1] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[2] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[3] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[4] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[5] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[6] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[7] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[8] HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[9] ZHANG Ying-tao, ZHANG Jie, ZHANG Rui, ZHANG Wen-qiang. Photorealistic Style Transfer Guided by Global Information [J]. Computer Science, 2022, 49(7): 100-105.
[10] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[11] ZHOU Hui, SHI Hao-chen, TU Yao-feng, HUANG Sheng-jun. Robust Deep Neural Network Learning Based on Active Sampling [J]. Computer Science, 2022, 49(7): 164-169.
[12] SU Dan-ning, CAO Gui-tao, WANG Yan-nan, WANG Hong, REN He. Survey of Deep Learning for Radar Emitter Identification Based on Small Sample [J]. Computer Science, 2022, 49(7): 226-235.
[13] YU Shu-hao, ZHOU Hui, YE Chun-yang, WANG Tai-zheng. SDFA:Study on Ship Trajectory Clustering Method Based on Multi-feature Fusion [J]. Computer Science, 2022, 49(6A): 256-260.
[14] WANG Jun-feng, LIU Fan, YANG Sai, LYU Tan-yue, CHEN Zhi-yu, XU Feng. Dam Crack Detection Based on Multi-source Transfer Learning [J]. Computer Science, 2022, 49(6A): 319-324.
[15] CHU Yu-chun, GONG Hang, Wang Xue-fang, LIU Pei-shun. Study on Knowledge Distillation of Target Detection Algorithm Based on YOLOv4 [J]. Computer Science, 2022, 49(6A): 337-344.
Full text



No Suggested Reading articles found!