Computer Science ›› 2020, Vol. 47 ›› Issue (7): 84-91.doi: 10.11896/jsjkx.190900006

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Survey of Visual Image Saliency Detection

YUAN Ye1,2,3, HE Xiao-ge1, ZHU Ding-kun4, WANG Fu-lee4, XIE Hao-ran5, WANG Jun1, WEI Ming-qiang1,2,3, GUO Yan-wen3   

  1. 1 College of Computer Science and Technology/College of Artificial Intelligence,Nanjing University of Aeronautics and Astronautics,Nanjing 210006,China
    2 MIIT Key Laboratory of Pattern Analysis and Machine Intelligence,Nanjing 210016,China
    3 State Key Laboratory for Novel Software Technology,Nanjing University,Nanjing 210023,China
    4 The Open University of Hong Kong,Hong Kong 999077,China
    5 The Education University of Hong Kong,Hong Kong 999077,China
  • Received:2019-09-02 Online:2020-07-15 Published:2020-07-16
  • About author:YUAN Ye,born in 1998,postgraduate.His main research interests include computer vision and so on.
    WEI Ming-qiang,born in 1985,Ph.D,associate professor.His main research interest is computer graphics with an emphasis on smart geometry proces-sing.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China (61502137),Fundamental Research Funds for the Central Universities (NJ2019010),Grant from State Key Laboratory for Novel Software Technology at Nanjing University (KFKT2018B20),Seed Fund for Early Career Scheme of the Dean’s Research Fund 2018-19 (SFG-10) of the Education University of Hong Kong and Interdisciplinary Research Scheme of the Dean’s Research Fund 2018-19 (FLASS/DRF/IDS-3) of the Education University of Hong Kong

Abstract: In today’s society where image data are exploding,how to use computer to efficiently acquire and process image information has become an important research topic.Under the inspiration of human visual attention mechanism,researchers have found that when this mechanism is introduced into machine image processing tasks,the efficiency of information extraction can be greatly improved,thus saving more limited computing resources.Visual image saliency detection is to use computers to simulate the human visual attention mechanism to calculate the importance of the information of each part in the image,which has been widely used in image segmentation,video compression,target detection,image indexing and other aspects,and has important research values.This paper summarizes and introduces the research situation of image saliency detection algorithms.Firstly,it takes information-driven sources as starting point to summarize the saliency detection model,and then analyzes several typical saliency detection algorithms.The models are divided into 3 categories according to whether they are based on learning models,which are based on non-learning models,based on traditional machine learning models and based on deep learning models.For the first category,the paper compares in more details the saliency detection algorithms based on local contrast and global contrast,and points out their respective advantages and disadvantages.For the latter two categories,this paper analyzes the application of machine learning algorithms and deep learning in saliency detection.Finally,this paper summarizes and compares the existing saliency detection algorithms and prospects the future development direction of the research in this aspect.

Key words: Deep learning, Machine learning, Salientregion detection, Visual attention mechanism, Visual saliency detection

CLC Number: 

  • TP391
[1]HAN J,NGAN K,LI M,et al.Unsupervised extraction of visual attention objects in color images[J].IEEE TCSV,2006,16 (1):409-416.
[2]KO B,NAM J.Object-of-interest image segmentation based on human attention and semantic region clustering[J].Jopt Soc Am,2006,23(10):409-414.
[3]HADIZADEH H,BAJIC I V.Saliency-aware video compression[J].IEEE Transactions on Image Processing,2014,23(1):19-33.
[4]LIU F,SHEN T S,LOU S L,et al.Deep network saliency detection based on global model and local optimization[J].Acta Optica Sinica,2017,37(12):1215005.
[5]ZHENG L,WANG S J,LIU Z Q,et al.Fast imageretrieval:Query pruning and early termination[J].IEEE Transactions on Mul-timedia,2015,17(5):648-659.
[6]DESIMONE R,DUNCAN J.Neural mechanisms of selectivevisual attention[J].Annual Review of Neuroscience,1995,18(1):193-222.
[7]ITTI L,KOCH C,NIEBUR.A model of saliency-based visualattention for rapid scene analysis[J].IEEE Transactions on Pat-tern Analysis and Machine Intelligence,1998,20(11):1254-1259.
[8]MA Y F,ZHANG H J.Contrast-based image attention analysisby using fuzzy growing[C]//Proceedings of the 11th ACM Con-ference on Multimedia.NewYork:ACM,2003:374-381.
[9]ACHANTA R,ESTRADA F,WILS P,et al.Salient region detection and segmentation[C]//Proceedings of the 6th Interna-tionalConference on Computer Vision Systems.Berlin:Springer-Verlag,2008:66-75.
[10]GOFERMAN S,ZELNIK-MAMOR L,TAL A.Context-aware saliency detection[C]//Proceedings of the 23rd International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2010:2376-2383.
[11]ACHANTA R,HEMAMI S,ESTRADA F,et al.Frequency-tuned salient region detection[C]//Proceedings of the 22nd Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2009:1597-1604.
[12]ZHAI Y,SHAH M.Visual attention detection in video se-quencesusing spatiotemporal cues[C]//Proceedings of the 14th ACM Conference on Multimedia.New York:ACM Press,2006:815-824.
[13]CHENG M M,ZHANG G X,MITRA N J,et al.Global contrast-based salient region detection[C]//Proceedings of the 24th International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2011:409-416.
[14]PERAZZI F,KRAHENBUL P,PRITCH Y,et al.Saliency fil-ters:Contrast based filtering for salient region detection[C] //Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition.United States:CVPR,2012:733-740.
[15]SHI K,WANG K,LU J,et al.PISA:Pixelwise Image Saliency by Aggregating Complementary Appearance Contrast Measures with Spatial Priors[C] //Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Guangzhou:IEEE Computer Society,2013:2115-2122.
[16]LIU T,SUN J,ZHENG N,et al.Learning to detect a salient ob-ject[C] //Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Xi’an:IEEE Computer Society,2007:1-8.
[17]RAHTU E,KANNALA J,SALO M,et al.Segmenting Salient Objects from Images and Videos[C]//Proceedings of European Conference on Computer Vision.Heraklion:ECCV,2010:366-379.
[18]JIANG H Z,YUAN Z J,CHENG M M,et al.Salient object detection:A discriminative regional feature integration approach[C]//Proceedings of the 26th International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2013:2083-2090.
[19]MAI L,NIU Y,LIU F.Saliency aggregation:a datadriven approach[C]//Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition.Portland,OR,USA:IEEE Computer Society,2013:1131-1138.
[20]LU H C,TONG N,ZHANG X N,et al.Co-Bootstrapping saliency[J].IEEE Transactions on Image Processing,2017,26(1):414-425.
[21]LI G,YU Y.Visual Saliency Based on Multiscale Deep Features[C] //Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Boston,MA,USA:IEEE Compu-ter Society,2015:5455-5463.
[22]ZHAO R,OUYANG W,LI H,et al.Saliency detection by multi-context deep learning[C] //Proceedings of 2015 IEEEConfe-rence on Computer Vision and Pattern Recognition.Boston,MA,USA:IEEE Computer Society,2015:1265-1274.
[23]WANG L,LU H,RUAN X,et al.Deep networks for saliency detection via local estimation and global search[C]//Procee-dings of the 28th International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2015:3183-3192.
[24]LI G,YU Y.Deep contrast learning for salient object detection[C]//Proceedings of the 29th International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2016:478-487.
[25]LIU N,HAN J.Dhsnet:Deep hierarchical saliency network for salient object detection[C]//Proceedings of the IEEE Confe-rence on Computer Vision and Pattern Recognition (CVPR).LasVegas,NV:IEEE Computer Society,2016:678-686.
[26]LEE G,TAI Y W,KIM J.Deep saliency with encoded low level distance map and high level features[C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Las Vegas,NV,USA:IEEE Computer Society,2016:660-668.
[27]HOU Q,CHENG M M,HU X,et al.Deeply supervised salient object detection with short connections[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Honolulu,HI:IEEE Computer Society,2017:5300-5309.
[28]ZHANG L,DAI J,LU H C,et al.A Bi-directional MessagePassing Model for Salient Object Detections[C]//Proceedings of The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Salt Lake City,UT,USA:IEEE Computer Society,2018:1741-1750.
[29]WU J Y.Review of Bottom-up Salient Object Detection[J].Computer Science,2019,46(3):49-52.
[1] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[2] LENG Dian-dian, DU Peng, CHEN Jian-ting, XIANG Yang. Automated Container Terminal Oriented Travel Time Estimation of AGV [J]. Computer Science, 2022, 49(9): 208-214.
[3] NING Han-yang, MA Miao, YANG Bo, LIU Shi-chang. Research Progress and Analysis on Intelligent Cryptology [J]. Computer Science, 2022, 49(9): 288-296.
[4] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[5] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[6] LI Yao, LI Tao, LI Qi-fan, LIANG Jia-rui, Ibegbu Nnamdi JULIAN, CHEN Jun-jie, GUO Hao. Construction and Multi-feature Fusion Classification Research Based on Multi-scale Sparse Brain Functional Hyper-network [J]. Computer Science, 2022, 49(8): 257-266.
[7] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[8] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[9] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[10] ZHANG Guang-hua, GAO Tian-jiao, CHEN Zhen-guo, YU Nai-wen. Study on Malware Classification Based on N-Gram Static Analysis Technology [J]. Computer Science, 2022, 49(8): 336-343.
[11] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[12] HE Qiang, YIN Zhen-yu, HUANG Min, WANG Xing-wei, WANG Yuan-tian, CUI Shuo, ZHAO Yong. Survey of Influence Analysis of Evolutionary Network Based on Big Data [J]. Computer Science, 2022, 49(8): 1-11.
[13] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[14] ZHOU Hui, SHI Hao-chen, TU Yao-feng, HUANG Sheng-jun. Robust Deep Neural Network Learning Based on Active Sampling [J]. Computer Science, 2022, 49(7): 164-169.
[15] SU Dan-ning, CAO Gui-tao, WANG Yan-nan, WANG Hong, REN He. Survey of Deep Learning for Radar Emitter Identification Based on Small Sample [J]. Computer Science, 2022, 49(7): 226-235.
Full text



No Suggested Reading articles found!