Computer Science

Computer Science ›› 2021, Vol. 48 ›› Issue (6A): 107-112.doi: 10.11896/jsjkx.201100116

• Image Processing & Multimedia Technology • Previous Articles     Next Articles

Saliency Detection Based on Eye Fixation Prediction and Boundary Optimization

LIU Xiang-yu1, JIAN Mu-wei1, LU Xiang-wei1, HE Wei-kai2, LI Xiao-feng3, YIN Yi-long3   

  1. 1 School of Computer Science and Technology,Shandong University of Finance and Economics,Jinan 250014,China
    2 School of Aeronautics,Shandong Jiaotong University,Jinan 250357,China
    3 School of Computer Science and Technology,Shandong Jianzhu University,Jinan 250000,China
  • Online:2021-06-10 Published:2021-06-17
  • About author:LIU Xiang-yu,master.His main researchinterests include image processing and visual saliency detection.
    JIAN Mu-wei,professor,doctoral supervisor,CCF computer vision committee,CCF multimedia committee,CCF machine learning and pattern recognition communications committee,etc.His main research interests include image processing,pattern recognition,multimedia computing,and so on.
  • Supported by:
    National Natural Science Foundation of China(61976123),Taishan Young Scholars Program of Shandong Province and Key Development Program for Basic Research of Shandong Province(ZR2020ZD44).

PDF (PC)

729

Abstract: Saliency detection is one of the most fundamental challenges in computer vision.Although the rapid development of deep learning has greatly improved the accuracy of saliency-detection results,the extraction of details of salient object is still unsatisfactory.Therefore,this paper proposes an edge refinement network based on eye-fixation prediction priori for salient object detection.Firstly,eye-fixation extraction is carried out on the original image and the extracted feature image is used as the visual priori of subsequent saliency detection.Secondly,the multi-attention mechanism of VGG16 network is used for feature extraction,and finally the feature image is refined to improve the quality of the saliency image.Experimental results show that,compared with other 6 state-of-the-art methods,the proposed method achieves better results in 3 open-accessed data sets(i.e.DUTS,ECSSD,HKU-IS).

Key words: Attention mechanism, Edge refinement, Eye-fixation prediction, Saliency detection

CLC Number: 

  • TP391
[1] LAURENT I,KOCH C,NIEBUR E.A model of saliency-based visual attention for rapid scene analysis[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2002,20(11):1254-1259.
[2] LE MEUR O,LE CALLET P,BARBA D,et al.A coherentcomputational approach to model bottom-up visual attention[J].IEEE TPAMI,2006,28(5):802-817.
[3] MATHE S,SMINCHISESCU C.Dynamic eye movement datasets and learnt saliency models for visual action recognition[C]//ECCV.2012.
[4] MATHE S,SMINCHISESCU C.Action from still image dataset and inverse optimal control to learn task specific visual scanpaths[C]//NIPS.2013.
[5] MATHE S,SMINCHISESCU C.Actions in the eye:Dynamic gaze datasets and learnt saliency models for visual recognition[J].IEEE TPAMI,2015,37(7):1408-1424.
[6] ZHANG L,TONG M H,MARKS T K,et al.SUN:A bayesian framework for saliency using natural statistics[J].Journal of Vision,2008,8(7):32-32.
[7] GAO D,VASCONCELOS N.Discriminant saliency for visualrecognition from cluttered scenes[C]//NIPS.2005.
[8] BRUCE N,TSOTSOS J.Saliency based on information maximization[C]//NIPS.2006.
[9] HAREL J,KOCH C,PERONA P.Graph-based visual saliency[C]//NIPS.2007.
[10] HOU X,ZHANG L.Saliency detection:A spectral residual approach[C]//CVPR.2007.
[11] JUDD T,EHINGER K,DURAND F,et al.Learning to predict where humans look[C]//ICCV.2009.
[12] KRUTHIVENTI S,GUDISA V,DHOLAKIYA J,et al.Saliency Unified:A Deep Architecture for simultaneous Eye Fixation Prediction and Salient Object Segmentation[C]//Computer Vision & Pattern Recognition.IEEE,2016.
[13] VIG E,DORR M,COX D.Large-scale optimization of hierarchical features for saliency prediction in natural images[C]//CVPR.2014.
[14] ZHOU Z,ZHANG L.AFSnet:Fixation Prediction in MovieScenes with Auxiliary Facial Saliency[C]//International Conference on Brain Inspired Cognitive Systems.Cham:Springer,2018.
[15] VOLODYMYR M,HEESS N,GRAVES A.Recurrent models of visual attention[C]//Advances in Neural Information Processing Systems.2014.
[16] ZHANG X.Progressive Attention Guided Recurrent Networkfor Salient Object Detection[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.IEEE,2018.
[17] LIU N,HAN J,YANG M H.PiCANet:Learning Pixel-WiseContextual Attention for Saliency Detection[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).IEEE,2018.
[18] WANG W,SHEN J.Deep Visual Attention Prediction[J].IEEE Transactions onImage Processing,2017(99):1-1.
[19] CORNIA M.Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model[J].IEEE Transactions on Image Processing,2016(27):5142-5154.
[20] DANDAN Z.MAFL:Multi-scale Adversarial Feature Learning for Saliency Detection[C]//2018 International Conference.2018.
[21] LINA W.Deep Group-Wise Fully Convolutional Network forCo-Saliency Detection With Graph Propagation[J].IEEE Transactions on Image Processing,2019(99).
[22] HECHT-NIELSEN R.Theory of the backpropagation neuralnetwork[J].Neural Networks,1988,1(Supplement-1):445-448.
[23] ZAKARIA L,KANNALA J.Context Aware Query Image Representation for Particular Object Retrieval[C]//Scandinavian Conference on Image Analsysi.Cham:Springer,2017:88-99.
[24] HU J,SHEN L,SUN G.Squeeze-and-Excitation Networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:7132-7141.
[25] WANG L,LU H,WANG Y,et al.Learning to detect salient objects with image-level supervision[C]//Proc.IEEE Conf.Comput.Vis.Pattern Recognit.(CVPR).2017:136-145.
[26] YAN Q,XU L,SHI J,et al.Hierarchical saliency detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2013:1155-1162.
[27] LI G,YU Y.Visual saliency based on multiscale deep features[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:5455-5463.
[28] ZHANG P,WANG D,LU H,et al.Amulet:Aggregating multi-level convolutional features for salient object detection[C]//IEEE International Conference on Computer Vision(ICCV 2017).Venice,Italy,2017:202-211.
[29] LI G,YU Y.Visual saliency detection based on multiscale deep cnn features[J].IEEE Transactions on Image Processing,2016,25(11):5012-5024.
[30] LUO Z,MISHRA A,ACHKAR A,et al.Non-local deep features for salient object detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE,2017:6593-6601.
[31] ZHANG P,WANG D,LU H,et al.Learning uncertain convolu-tional features for accurate saliency detection[C]//IEEE Inter-national Conference on Computer Vision (ICCV 2017).Venice,
Italy,2017:212-221.
[32] LI G,YU Y.Deep contrast learning for salient object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:478-487.
[33] LIU N,HAN J.Dhsnet:Deep hierarchical saliency network for salient object detection[C]//Proceedings of the IEEE Confe-rence on Computer Vision and Pattern Recognition.2016:678-686.
[34] SIMONYAN K,ZISSERMAN A.Very deep convolutionalnetworks for large-scale image recognition[J].arXiv:1409.1556,2014.
[35] RONNEBERGER O,FISCHER P,BROX T.Unet Convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computingand Computer-assisted Intervention.Springer,2015:234-241.
[36] ZHAO T,WU X.Pyramid feature attention network for saliency detection[C]//IEEE Conf.Comput.Vis.Pattern Recog..2019:3085-3094.
[37] QIN X.BASNet:Boundary-Aware Salient Object Detection[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).IEEE,2019.
[38] WANG W.Salient Object Detection Driven by Fixation Prediction[C]//IEEE CVPR.IEEE,2018.
[1] ZHOU Fang-quan, CHENG Wei-qing. Sequence Recommendation Based on Global Enhanced Graph Neural Network [J]. Computer Science, 2022, 49(9): 55-63.
[2] DAI Yu, XU Lin-feng. Cross-image Text Reading Method Based on Text Line Matching [J]. Computer Science, 2022, 49(9): 139-145.
[3] ZHOU Le-yuan, ZHANG Jian-hua, YUAN Tian-tian, CHEN Sheng-yong. Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion [J]. Computer Science, 2022, 49(9): 155-161.
[4] XIONG Li-qin, CAO Lei, LAI Jun, CHEN Xi-liang. Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization [J]. Computer Science, 2022, 49(9): 172-182.
[5] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[6] WANG Ming, PENG Jian, HUANG Fei-hu. Multi-time Scale Spatial-Temporal Graph Neural Network for Traffic Flow Prediction [J]. Computer Science, 2022, 49(8): 40-48.
[7] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[8] ZHU Cheng-zhang, HUANG Jia-er, XIAO Ya-long, WANG Han, ZOU Bei-ji. Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism [J]. Computer Science, 2022, 49(8): 113-119.
[9] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[10] YAN Jia-dan, JIA Cai-yan. Text Classification Method Based on Information Fusion of Dual-graph Neural Network [J]. Computer Science, 2022, 49(8): 230-236.
[11] ZHANG Ying-tao, ZHANG Jie, ZHANG Rui, ZHANG Wen-qiang. Photorealistic Style Transfer Guided by Global Information [J]. Computer Science, 2022, 49(7): 100-105.
[12] ZENG Zhi-xian, CAO Jian-jun, WENG Nian-feng, JIANG Guo-quan, XU Bin. Fine-grained Semantic Association Video-Text Cross-modal Entity Resolution Based on Attention Mechanism [J]. Computer Science, 2022, 49(7): 106-112.
[13] XU Ming-ke, ZHANG Fan. Head Fusion:A Method to Improve Accuracy and Robustness of Speech Emotion Recognition [J]. Computer Science, 2022, 49(7): 132-141.
[14] MENG Yue-bo, MU Si-rong, LIU Guang-hui, XU Sheng-jun, HAN Jiu-qiang. Person Re-identification Method Based on GoogLeNet-GMP Based on Vector Attention Mechanism [J]. Computer Science, 2022, 49(7): 142-147.
[15] JIN Fang-yan, WANG Xiu-li. Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM [J]. Computer Science, 2022, 49(7): 179-186.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.