Computer Science ›› 2021, Vol. 48 ›› Issue (6): 103-109.doi: 10.11896/jsjkx.200600068

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Learning Global Guided Progressive Feature Aggregation Lightweight Network for Salient Object Detection

PAN Ming-yuan, SONG Hui-hui, ZHANG Kai-hua, LIU Qing-shan   

  1. Collaborative Innovation Center on Atmospheric Environment and Equipment Technology,Nanjing University of Information Science and Technology,Nanjing 210044,China
    Jiangsu Key Laboratory of Big Data Analysis Technology,Nanjing University of Information Science and Technology,Nanjing 210044,China
  • Received:2020-06-12 Revised:2020-09-22 Online:2021-06-15 Published:2021-06-03
  • About author:PAN Ming-yuan,born in 1995,postgraduate.His main research interest is salient object detection.(pan_mingyuan@foxmail.com)
    SONG Hui-hui,born in 1986,Ph.D,professor,is a member of China Computer Federation.Her main research interests include saliency detection and image super-resolution.
  • Supported by:
    National Major Project of China for New Generation of AI (2018AAA0100400),National Natural Science Foundation of China(61872189,61876088) and Natural Science Foundation of Jiangsu Province(BK20191397,BK20170040).

Abstract: To solve the problems of insufficient feature fusion and redundant models in salient object detection algorithms,this paper proposes a novel globally guided progressive feature aggregation network for lightweight salient object detection.Firstly,the lightweight feature extraction network MobileNetV3 is used to extract different levels of features of the image.Then,the lightweight multi-scale receptive field enhancement module is applied to further enhance the global representation of the highestlevel feature extracted by MobileNetV3.Finally,the progressive feature aggregation module is utilized to progressively fuse high-level and low-level features from top to bottom and the common cross entropy loss function is used to optimize these fused features in multiple stages,so as to obtain the saliency maps from coarse to fine.The whole network is an absolute end-to-end framework without any pre-processing and post-processing.Extensive experiments on six benchmark datasets demonstrate the superiority of the proposed method against other 10 methods in terms of metrics such as PR Curve,F-measure,S-measure and MAE.At the same time,the model is only about 10MB and can run at a speed of 46FPS on a GTX2080Ti GPU when processing a 400×300 image.

Key words: Convolutional neural network, Fast, Feature fusion, Lightweight, Salient object detection

CLC Number: 

  • TP391
[1]WANG Y,XU X F.Image Segmentation Based on Saliency and Pulse Coupled Neural Network[J].Computer Science,2018,45(7):259-263.
[2]DONOSER M,URSCHLER M,HIRZER M,et al.Saliencydriven total variation segmentation[C]//2009 IEEE 12th International Conference on Computer Vision.IEEE,2009:817-824.
[3]ZHANG D,MENG D,ZHAO L,et al.Bridging saliency detection to weakly supervised object[J].arXiv:1703.01290,2017.
[4]ZHANG Z F,WU Z M,DU L,et al.Video Saliency Detection Based on Compressed Domain Coding Length[J].Computer Science,2017,44(10):312-317.
[5]FAN D P,WANG W,CHENG M M,et al.Shifting more attention to video salient object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2019:8554-8564.
[6]HONG S,YOU T,KWAK S,et al.Online tracking by learning discriminative saliency map with convolutional neural network[C]//International Conference on Machine Learning.2015:597-606.
[7]YANG C,ZHANG L,LU H,et al.Saliency detection via graph-based manifold ranking[C]//Proceedings of the IEEEConfe-rence on Computer Vision and Pattern Recognition.2013:3166-3173.
[8]ZHU W,LIANG S,WEI Y,et al.Saliency optimization from robust background detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2014:2814-2821.
[9]CHENG M M,MITRA N J,HUANG X,et al.Global contrast based salient region detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2014,37(3):569-582.
[10]HOU Q,CHENG M M,HU X,et al.Deeply supervised salient object detection with short connections[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:3203-3212.
[11]TANG Y,WU X,BU W.Deeply-supervised recurrent convolutional neural network for saliency detection[C]//Proceedings of the 24th ACM International Conference on Multimedia.2016:397-401.
[12]ZHANG P,WANG D,LU H,et al.Amulet:Aggregating multi-level convolutional features for salient object detection[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:202-211.
[13]LONG J,SHELHAMER E,DARRELL T.Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:3431-3440.
[14]LI X,ZHAO L,WEI L,et al.Deepsaliency:Multi-task deep neural network model for salient object detection[J].IEEETransa-ctions on Image Processing,2016,25(8):3919-3930.
[15]CHEN S,TAN X,WANG B,et al.Reverse attention for salient object detection[C]//Proceedings of the European Conference on Computer Vision(ECCV).2018:234-250.
[16]WANG W,ZHAO S,SHEN J,et al.Salient object detectionwith pyramid attention and salient edges[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2019:1448-1457.
[17]HOWARD A,SANDLER M,CHU G,et al.Searching for mobilenetv3[C]//Proceedings of the IEEE International Confe-rence on Computer Vision.2019:1314-1324.
[18]WANG W,LAI Q,FU H,et al.Salient object detection in the deep learning era:An in-depth survey[J].arXiv:1904.09146,2019.
[19]BORJI A,CHENG M M,JIANG H,et al.Salient object detection:A benchmark[J].IEEE Transactions on Image Processing,2015,24(12):5706-5722.
[20]XIE S,TU Z.Holistically-nested edge detection[C]//Procee-dings of the IEEE International Conference on Computer Vision.2015:1395-1403.
[21]WANG T,BORJI A,ZHANG L,et al.A stagewise refinement model for detecting salient objects in images[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:4019-4028.
[22]WU Z,SU L,HUANG Q.Cascaded partial decoder for fast and accurate salient object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2019:3907-3916.
[23]ISLAM M A,KALASH M,ROCHAN M,et al.Salient Object Detection using a Context-Aware Refinement Network[C]//BMVC.2017.
[24]DENG Z,HU X,ZHU L,et al.R3net:Recurrent residual refinement network for saliency detection[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence.AAAI Press,2018:684-690.
[25]WANG T,ZHANG L,WANG S,et al.Detect globally,refine locally:A novel approach to saliency detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:3127-3135.
[26]LIU N,HAN J,YANG M H.Picanet:Learning pixel-wise contextual attention for saliency detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:3089-3098.
[27]LIU S,HUANG D.Receptive field block net for accurate andfast object detection[C]//Proceedings of the European Confe-rence on Computer Vision(ECCV).2018:385-400.
[28]HU J,SHEN L,SUN G.Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:7132-7141.
[29]HOU Q,ZHANG L,CHENG M M,et al.Strip Pooling:Rethinking Spatial Pooling for Scene Parsing[J].arXiv:2003.13328,2020.
[30]WANG L,LU H,WANG Y,et al.Learning to detect salient objects with image-level supervision[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:136-145.
[31]DENG J,DONG W,SOCHER R,et al.Imagenet:A large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2009:248-255.
[32]KINGMA D P,BA J.Adam:A method for stochastic optimization[J].arXiv:1412.6980,2014.
[33]LI X,LU H,ZHANG L,et al.Saliency detection via dense and sparse reconstruction[C]//Proceedings of the IEEE International Conference on Computer Vision.2013:2976-2983.
[34]LI G,YU Y.Visual saliency based on multiscale deep features[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:5455-5463.
[35]LI Y,HOU X,KOCH C,et al.The secrets of salient object segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2014:280-287.
[36]MOVAHEDI V,ELDER J H.Design and perceptual validation
of performance measures for salient object segmentation[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops.IEEE,2010:49-56.
[37]YANG C,ZHANG L,LU H,et al.Saliency detection via graph-based manifold ranking[C]//Proceedings of the IEEEConfe-rence on Computer Vision and Pattern Recognition.2013:3166-3173.
[38]MARTIN D,FOWLKES C,TAL D,et al.A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//Proceedings Eighth IEEE International Conference on Computer Vision(ICCV 2001).IEEE,2001:416-423.
[39]FAN D P,CHENG M M,LIU Y,et al.Structure-measure:Anew way to evaluate foreground maps[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:4548-4557.
[40]LI G,YU Y.Deep contrast learning for salient object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:478-487.
[41]LUO Z,MISHRA A,ACHKAR A,et al.Non-local deep features for salient object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:6609-6617.
[42]ZHANG P,WANG D,LU H,et al.Learning uncertain convolutional features for accurate saliency detection[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:212-221.
[43]FENG M,LU H,DING E.Attentive feedback network forboundary-aware salient object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2019:1623-1632.
[1] ZHOU Le-yuan, ZHANG Jian-hua, YUAN Tian-tian, CHEN Sheng-yong. Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion [J]. Computer Science, 2022, 49(9): 155-161.
[2] CHEN Yong-quan, JIANG Ying. Analysis Method of APP User Behavior Based on Convolutional Neural Network [J]. Computer Science, 2022, 49(8): 78-85.
[3] ZHU Cheng-zhang, HUANG Jia-er, XIAO Ya-long, WANG Han, ZOU Bei-ji. Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism [J]. Computer Science, 2022, 49(8): 113-119.
[4] ZHANG Ying-tao, ZHANG Jie, ZHANG Rui, ZHANG Wen-qiang. Photorealistic Style Transfer Guided by Global Information [J]. Computer Science, 2022, 49(7): 100-105.
[5] DAI Zhao-xia, LI Jin-xin, ZHANG Xiang-dong, XU Xu, MEI Lin, ZHANG Liang. Super-resolution Reconstruction of MRI Based on DNGAN [J]. Computer Science, 2022, 49(7): 113-119.
[6] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[7] LIU Yue-hong, NIU Shao-hua, SHEN Xian-hao. Virtual Reality Video Intraframe Prediction Coding Based on Convolutional Neural Network [J]. Computer Science, 2022, 49(7): 127-131.
[8] XU Ming-ke, ZHANG Fan. Head Fusion:A Method to Improve Accuracy and Robustness of Speech Emotion Recognition [J]. Computer Science, 2022, 49(7): 132-141.
[9] WU Zi-bin, YAN Qiao. Projected Gradient Descent Algorithm with Momentum [J]. Computer Science, 2022, 49(6A): 178-183.
[10] ZHANG Jia-hao, LIU Feng, QI Jia-yin. Lightweight Micro-expression Recognition Architecture Based on Bottleneck Transformer [J]. Computer Science, 2022, 49(6A): 370-377.
[11] ZHU Wen-tao, LAN Xian-chao, LUO Huan-lin, YUE Bing, WANG Yang. Remote Sensing Aircraft Target Detection Based on Improved Faster R-CNN [J]. Computer Science, 2022, 49(6A): 378-383.
[12] WANG Jian-ming, CHEN Xiang-yu, YANG Zi-zhong, SHI Chen-yang, ZHANG Yu-hang, QIAN Zheng-kun. Influence of Different Data Augmentation Methods on Model Recognition Accuracy [J]. Computer Science, 2022, 49(6A): 418-423.
[13] CHEN Yong-ping, ZHU Jian-qing, XIE Yi, WU Han-xiao, ZENG Huan-qiang. Real-time Helmet Detection Algorithm Based on Circumcircle Radius Difference Loss [J]. Computer Science, 2022, 49(6A): 424-428.
[14] SUN Jie-qi, LI Ya-feng, ZHANG Wen-bo, LIU Peng-hui. Dual-field Feature Fusion Deep Convolutional Neural Network Based on Discrete Wavelet Transformation [J]. Computer Science, 2022, 49(6A): 434-440.
[15] SHAO Xin-xin. TI-FastText Automatic Goods Classification Algorithm [J]. Computer Science, 2022, 49(6A): 206-210.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!