Computer Science ›› 2024, Vol. 51 ›› Issue (6A): 230300204-8.doi: 10.11896/jsjkx.230300204

• Image Processing & Multimedia Technolog • Previous Articles     Next Articles

Lightweight Image Semantic Segmentation Based on Attention Mechanism and Densely AdjacentPrediction

WANG Guogang, DONG Zhihao   

  1. College of Physics and Electronic Engineering,Shanxi University,Taiyuan 030006,China
  • Published:2024-06-06
  • About author:WANG Guogang,born in 1977,Ph.D,associate professor,is a member of CCF(No.K7194M).His main research interests include the image processing and computer vision,and artificial intelligence.
  • Supported by:
    National Natural Science Foundation of China(11804209) and Natural Science Foundation of Shanxi Province,China(201901D111031,201901D211173).

Abstract: A novel algorithm named as lightweight image semantic segmentation based on attention mechanism and densely adjacent prediction is proposed to avoid the disadvantages of the difficulty in highlighting important channel features for atrous spatial pyramid pooling module,higher computational complexity and lacking of sufficient detailed information for the high level semantic feature map generated by the decoder in DeepLabv3+ algorithm.The lightweight MobileNetV2 is regarded as the backbone network to reduce model parameters.After the multi-scale information is extracted by the channel atrous spatial pyramid pooling,each channel of the feature map is weighted to reinforce the learning of important channel features.Moreover,the segmentation results are refined since densely adjacent prediction is utilized to combine high-level and low-level features.Experiments are performed on the PASCAL VOC 2012 augmented dataset,and the experimental results show that both mean Intersection over union and mean pixel accuracy of the proposed method are higher than the state-of-the-art algorithms.Compared with DeepLabv3+,the parameters and calculation amount are decreased by 184.82×106 and 90.83GFLOPs respectively.The proposed algorithm not only improves the segmentation accuracy,but also reduces the computation cost compared to the baseline algorithm.

Key words: Deep learning, Semantic segmentation, DeepLabv3+, Attention mechanism

CLC Number: 

  • TP391
[1]CAI Y F,DAI L,WANG H,et al.Multi-Target Pan-Class In-trinsic Relevance Driven Model for Improving Semantic Segmentation in Autonomous Driving[J].IEEE Transactions on Image Processing,2021,30:9069-9084.
[2]ZHOU W,BERRIO J S,WORRAL S,et al.Automated Evalua-tion of Semantic Segmentation Robustness for Autonomous Driving[J].IEEE Transactions on Intelligent Transportation Systems,2020,21(5):1951-1963.
[3]YI D W,FANG H,HUA Y N,et al.Improving Synthetic to Realistic Semantic Segmentation With Parallel Generative Ensembles for Autonomous Urban Driving[J].IEEE Transactions on Cognitive and Developmental Systems,2022,14(4):1496-1506.
[4]YIN P S,YUAN R,CHENG Y M,et al.Deep Guidance Network for Biomedical Image Segmentation[J].IEEE Access,2020,8:116106-116116.
[5]ZHANG M,LI X,XU M J,et al.Automated Semantic Segmentation of Red Blood Cells for Sickle Cell Disease[J].IEEE Journal of Biomedical and Health Informatics,2020,24(11):3095-3102.
[6]GAO Z J,HE Y,LI Y.A Novel Lightweight Swin-Unet Network for Semantic Segmentation of COVID-19 Lesion in CT Images[J].IEEE Access,2023,11:950-962.
[7]IBRAHIM M,AKHTAR N,WISE M,et al.Annotation Tool and Urban Dataset for 3D Point Cloud Semantic Segmentation[J].IEEE Access,2021,9:35984-35996.
[8]SHI W J,XU J W,ZHU D C,et al.RGB-D Semantic Segmentation and Label-Oriented Voxelgrid Fusion for Accurate 3D Semantic Mapping[J].IEEE Transactions on Circuits and Systems for Video Technology,2022,32(1):183-197.
[9]YANG J F,ZHOU B C,QIU H D,et al.MLFNet-Point Cloud Semantic Segmentation Convolution Network Based on Multi-Scale Feature Fusion[J].IEEE Access,2021,9:44950-44962.
[10]MINAEE S,BOYKOV Y,PORIKLI F,et al.Image Segmentation Using Deep Learning:A Survey[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2022,44(7):3523-3542.
[11]LONG J,SHELHAMER E,DARRELL T,et al.Fully convolutional networks for semantic segmentation[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Boston,MA,USA,2015:3431-3440.
[12]BADRINARAYANAN V,KENDALL A,CIPOLLA R.Segnet:a deep convolutional encoder-decoder architecture for image segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(12):2481-2495.
[13]RONNEBERGER O,FISCHER P,BROX T.U-Net:Convolu-tional Networks for Biomedical Image Segmentation[C]//International Conference on Medical image computing and computer-assisted intervention.Cham:Springer,2015:234-241.
[14]ZHAO H S,SHI J P,QI X J,et al.Pyramid Scene Parsing Network[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Honolulu,HI,USA,2017:6230-6239.
[15]CHEN L C,PAPANDREOU G,KOKKINOS I,et al.Deeplab:semantic image segmentation with deep convolutional nets,atrous convolution,and fully connected CRFs[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,40(4):834-848.
[16]CHEN L C,PAPANDREOU G,SCHROFF F,et al.Rethinking atrous convolution for semantic image segmentation[EB/OL].(2017-12-05)[2023-02-24].https://arxiv.org/abs/1706.05587.
[17]CHEN L C,PAPANDREOU G,KOKKINOS I,et al.Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs[EB/OL].(2014-12-22)[2023-02-24].https://arxiv.org/abs/1412.7062.
[18]FU J,LIU J,TIAN H J,et al.Dual Attention Network for SceneSegmentation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Long Beach,CA,USA,2019:3141-3149.
[19]WANG X L,GIRSHICK R,GUPTA A,et al.Non-local neural networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City,UT,USA,2018:7794-803.
[20]YIN M L,YAO Z H,CAO Y,et al.Disentangled non-local neu-ral networks[C]//Proceedings of the European Conference on Computer Vision.2020:191-207.
[21]HUANG Z L,WANG X G,WEI C C,et al.CCNet:Criss-Cross Attention for Semantic Segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2023,45(6):6896-6908.
[22]CHEN L C,ZHU Y,PAPANDREOU G,et al.Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation[C]//Proceedings of the European Conference on Computer Vision(ECCV).2018:801-818.
[23]CHOLLET F.Xception:Deep Learning with Depthwise Separable Convolutions[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Honolulu,HI,USA,2017:1800-1807.
[24]SANDLER M,HOWARD A,ZHU M,et al.MobileNetV2:Inverted Residuals and Linear Bottlenecks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City,UT,USA,2018:4510-4520.
[25]ZHANG Z L,ZHANG X Y,PENG C,et al.ExFuse:Enhancing Feature Fusion for Semantic Segmentation[C]//European Conference on Computer Vision.Charm:Springer,2018:273-288.
[26]SHI W,CABALLERO J,HUSZAR F,et al.Real-Time SingleImage and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Las Vegas,NV,USA,2016:1874-1883.
[27]PENG C,ZHANG X Y,YU G,et al.Large Kernel Matters—Improve Semantic Segmentation by Global ConvolutionalNetwork[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Honolulu,HI,USA,2017:1743-1751.
[1] HUANG Haixin, CAI Mingqi, WANG Yuyao. Review of Point Cloud Semantic Segmentation Based on Graph Convolutional Neural Networks [J]. Computer Science, 2024, 51(6A): 230400196-7.
[2] LIU Xiaohu, CHEN Defu, LI Jun, ZHOU Xuwen, HU Shan, ZHOU Hao. Speaker Verification Network Based on Multi-scale Convolutional Encoder [J]. Computer Science, 2024, 51(6A): 230700083-6.
[3] WANG Yingjie, ZHANG Chengye, BAI Fengbo, WANG Zumin. Named Entity Recognition Approach of Judicial Documents Based on Transformer [J]. Computer Science, 2024, 51(6A): 230500164-9.
[4] ZHU Yuliang, LIU Juntao, RAO Ziyun, ZHANG Yi, CAO Wanhua. Knowledge Reasoning Model Combining HousE with Attention Mechanism [J]. Computer Science, 2024, 51(6A): 230600209-8.
[5] LIANG Fang, XU Xuyao, ZHAO Kailong, ZHAO Xuanfeng, ZHANG Guijun. Remote Template Detection Algorithm and Its Application in Protein Structure Prediction [J]. Computer Science, 2024, 51(6A): 230600225-7.
[6] BAI Yu, WANG Xinzhe. Study on Hypernymy Recognition Based on Combined Training of Attention Mechanism and Prompt Learning [J]. Computer Science, 2024, 51(6A): 230700226-5.
[7] PENG Bo, LI Yaodong, GONG Xianfu, LI Hao. Method for Entity Relation Extraction Based on Heterogeneous Graph Neural Networks and TextSemantic Enhancement [J]. Computer Science, 2024, 51(6A): 230700071-5.
[8] ZHANG Tianchi, LIU Yuxuan. Research Progress of Underwater Image Processing Based on Deep Learning [J]. Computer Science, 2024, 51(6A): 230400107-12.
[9] SUN Jifei, JIA Kebin. Classification and Detection Algorithm of Ground-based Cloud Images Based on Multi-scale Features [J]. Computer Science, 2024, 51(6A): 230400041-6.
[10] ZHANG Le, YU Ying, GE Hao. Mural Inpainting Based on Fast Fourier Convolution and Feature Pruning Coordinate Attention [J]. Computer Science, 2024, 51(6A): 230400083-9.
[11] WU Yibo, HAO Yingguang, WANG Hongyu. Rice Defect Segmentation Based on Dual-stream Convolutional Neural Networks [J]. Computer Science, 2024, 51(6A): 230600107-8.
[12] HOU Linhao, LIU Fan. Remote Sensing Image Fusion Combining Multi-scale Convolution Blocks and Dense Convolution Blocks [J]. Computer Science, 2024, 51(6A): 230400110-6.
[13] HUANG Yuanhang, BIAN Shan, WANG Chuntao. Gaussian Enhancement Module for Reinforcing High-frequency Details in Camera ModelIdentification [J]. Computer Science, 2024, 51(6A): 230700125-5.
[14] SUN Yang, DING Jianwei, ZHANG Qi, WEI Huiwen, TIAN Bowen. Study on Super-resolution Image Reconstruction Using Residual Feature Aggregation NetworkBased on Attention Mechanism [J]. Computer Science, 2024, 51(6A): 230600039-6.
[15] QUE Yue, GAN Menghan, LIU Zhiwei. Object Detection with Receptive Field Expansion and Multi-branch Aggregation [J]. Computer Science, 2024, 51(6A): 230600151-6.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!