Computer Science ›› 2020, Vol. 47 ›› Issue (7): 84-91.doi: 10.11896/jsjkx.190900006

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Survey of Visual Image Saliency Detection

YUAN Ye1,2,3, HE Xiao-ge1, ZHU Ding-kun4, WANG Fu-lee4, XIE Hao-ran5, WANG Jun1, WEI Ming-qiang1,2,3, GUO Yan-wen3   

  1. 1 College of Computer Science and Technology/College of Artificial Intelligence,Nanjing University of Aeronautics and Astronautics,Nanjing 210006,China
    2 MIIT Key Laboratory of Pattern Analysis and Machine Intelligence,Nanjing 210016,China
    3 State Key Laboratory for Novel Software Technology,Nanjing University,Nanjing 210023,China
    4 The Open University of Hong Kong,Hong Kong 999077,China
    5 The Education University of Hong Kong,Hong Kong 999077,China
  • Received:2019-09-02 Online:2020-07-15 Published:2020-07-16
  • About author:YUAN Ye,born in 1998,postgraduate.His main research interests include computer vision and so on.
    WEI Ming-qiang,born in 1985,Ph.D,associate professor.His main research interest is computer graphics with an emphasis on smart geometry proces-sing.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China (61502137),Fundamental Research Funds for the Central Universities (NJ2019010),Grant from State Key Laboratory for Novel Software Technology at Nanjing University (KFKT2018B20),Seed Fund for Early Career Scheme of the Dean’s Research Fund 2018-19 (SFG-10) of the Education University of Hong Kong and Interdisciplinary Research Scheme of the Dean’s Research Fund 2018-19 (FLASS/DRF/IDS-3) of the Education University of Hong Kong

Abstract: In today’s society where image data are exploding,how to use computer to efficiently acquire and process image information has become an important research topic.Under the inspiration of human visual attention mechanism,researchers have found that when this mechanism is introduced into machine image processing tasks,the efficiency of information extraction can be greatly improved,thus saving more limited computing resources.Visual image saliency detection is to use computers to simulate the human visual attention mechanism to calculate the importance of the information of each part in the image,which has been widely used in image segmentation,video compression,target detection,image indexing and other aspects,and has important research values.This paper summarizes and introduces the research situation of image saliency detection algorithms.Firstly,it takes information-driven sources as starting point to summarize the saliency detection model,and then analyzes several typical saliency detection algorithms.The models are divided into 3 categories according to whether they are based on learning models,which are based on non-learning models,based on traditional machine learning models and based on deep learning models.For the first category,the paper compares in more details the saliency detection algorithms based on local contrast and global contrast,and points out their respective advantages and disadvantages.For the latter two categories,this paper analyzes the application of machine learning algorithms and deep learning in saliency detection.Finally,this paper summarizes and compares the existing saliency detection algorithms and prospects the future development direction of the research in this aspect.

Key words: Salientregion detection, Visual saliency detection, Deep learning, Machine learning, Visual attention mechanism

CLC Number: 

  • TP391
[1] HAN J,NGAN K,LI M,et al.Unsupervised extraction of visual attention objects in color images[J].IEEE TCSV,2006,16 (1):409-416.
[2] KO B,NAM J.Object-of-interest image segmentation based on human attention and semantic region clustering[J].Jopt Soc Am,2006,23(10):409-414.
[3] HADIZADEH H,BAJIC I V.Saliency-aware video compression[J].IEEE Transactions on Image Processing,2014,23(1):19-33.
[4] LIU F,SHEN T S,LOU S L,et al.Deep network saliency detection based on global model and local optimization[J].Acta Optica Sinica,2017,37(12):1215005.
[5] ZHENG L,WANG S J,LIU Z Q,et al.Fast imageretrieval:Query pruning and early termination[J].IEEE Transactions on Mul-timedia,2015,17(5):648-659.
[6] DESIMONE R,DUNCAN J.Neural mechanisms of selectivevisual attention[J].Annual Review of Neuroscience,1995,18(1):193-222.
[7] ITTI L,KOCH C,NIEBUR.A model of saliency-based visualattention for rapid scene analysis[J].IEEE Transactions on Pat-tern Analysis and Machine Intelligence,1998,20(11):1254-1259.
[8] MA Y F,ZHANG H J.Contrast-based image attention analysisby using fuzzy growing[C]//Proceedings of the 11th ACM Con-ference on Multimedia.NewYork:ACM,2003:374-381.
[9] ACHANTA R,ESTRADA F,WILS P,et al.Salient region detection and segmentation[C]//Proceedings of the 6th Interna-tionalConference on Computer Vision Systems.Berlin:Springer-Verlag,2008:66-75.
[10] GOFERMAN S,ZELNIK-MAMOR L,TAL A.Context-aware saliency detection[C]//Proceedings of the 23rd International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2010:2376-2383.
[11] ACHANTA R,HEMAMI S,ESTRADA F,et al.Frequency-tuned salient region detection[C]//Proceedings of the 22nd Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2009:1597-1604.
[12] ZHAI Y,SHAH M.Visual attention detection in video se-quencesusing spatiotemporal cues[C]//Proceedings of the 14th ACM Conference on Multimedia.New York:ACM Press,2006:815-824.
[13] CHENG M M,ZHANG G X,MITRA N J,et al.Global contrast-based salient region detection[C]//Proceedings of the 24th International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2011:409-416.
[14] PERAZZI F,KRAHENBUL P,PRITCH Y,et al.Saliency fil-ters:Contrast based filtering for salient region detection[C] //Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition.United States:CVPR,2012:733-740.
[15] SHI K,WANG K,LU J,et al.PISA:Pixelwise Image Saliency by Aggregating Complementary Appearance Contrast Measures with Spatial Priors[C] //Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Guangzhou:IEEE Computer Society,2013:2115-2122.
[16] LIU T,SUN J,ZHENG N,et al.Learning to detect a salient ob-ject[C] //Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Xi’an:IEEE Computer Society,2007:1-8.
[17] RAHTU E,KANNALA J,SALO M,et al.Segmenting Salient Objects from Images and Videos[C]//Proceedings of European Conference on Computer Vision.Heraklion:ECCV,2010:366-379.
[18] JIANG H Z,YUAN Z J,CHENG M M,et al.Salient object detection:A discriminative regional feature integration approach[C]//Proceedings of the 26th International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2013:2083-2090.
[19] MAI L,NIU Y,LIU F.Saliency aggregation:a datadriven approach[C]//Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition.Portland,OR,USA:IEEE Computer Society,2013:1131-1138.
[20] LU H C,TONG N,ZHANG X N,et al.Co-Bootstrapping saliency[J].IEEE Transactions on Image Processing,2017,26(1):414-425.
[21] LI G,YU Y.Visual Saliency Based on Multiscale Deep Features[C] //Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Boston,MA,USA:IEEE Compu-ter Society,2015:5455-5463.
[22] ZHAO R,OUYANG W,LI H,et al.Saliency detection by multi-context deep learning[C] //Proceedings of 2015 IEEEConfe-rence on Computer Vision and Pattern Recognition.Boston,MA,USA:IEEE Computer Society,2015:1265-1274.
[23] WANG L,LU H,RUAN X,et al.Deep networks for saliency detection via local estimation and global search[C]//Procee-dings of the 28th International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2015:3183-3192.
[24] LI G,YU Y.Deep contrast learning for salient object detection[C]//Proceedings of the 29th International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2016:478-487.
[25] LIU N,HAN J.Dhsnet:Deep hierarchical saliency network for salient object detection[C]//Proceedings of the IEEE Confe-rence on Computer Vision and Pattern Recognition (CVPR).LasVegas,NV:IEEE Computer Society,2016:678-686.
[26] LEE G,TAI Y W,KIM J.Deep saliency with encoded low level distance map and high level features[C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Las Vegas,NV,USA:IEEE Computer Society,2016:660-668.
[27] HOU Q,CHENG M M,HU X,et al.Deeply supervised salient object detection with short connections[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Honolulu,HI:IEEE Computer Society,2017:5300-5309.
[28] ZHANG L,DAI J,LU H C,et al.A Bi-directional MessagePassing Model for Salient Object Detections[C]//Proceedings of The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Salt Lake City,UT,USA:IEEE Computer Society,2018:1741-1750.
[29] WU J Y.Review of Bottom-up Salient Object Detection[J].Computer Science,2019,46(3):49-52.
[1] WANG Rui-ping, JIA Zhen, LIU Chang, CHEN Ze-wei, LI Tian-rui. Deep Interest Factorization Machine Network Based on DeepFM [J]. Computer Science, 2021, 48(1): 226-232.
[2] YU Wen-jia, DING Shi-fei. Conditional Generative Adversarial Network Based on Self-attention Mechanism [J]. Computer Science, 2021, 48(1): 241-246.
[3] TONG Xin, WANG Bin-jun, WANG Run-zheng, PAN Xiao-qin. Survey on Adversarial Sample of Deep Learning Towards Natural Language Processing [J]. Computer Science, 2021, 48(1): 258-267.
[4] LI Yin, LI Bi-xin. Memory Leak Test Acceleration Based on Script Prediction and Reconstruction [J]. Computer Science, 2020, 47(9): 31-39.
[5] DING Yu, WEI Hao, PAN Zhi-song, LIU Xin. Survey of Network Representation Learning [J]. Computer Science, 2020, 47(9): 52-59.
[6] HE Xin, XU Juan, JIN Ying-ying. Action-related Network:Towards Modeling Complete Changeable Action [J]. Computer Science, 2020, 47(9): 123-128.
[7] YE Ya-nan, CHI Jing, YU Zhi-ping, ZHAN Yu-liand ZHANG Cai-ming. Expression Animation Synthesis Based on Improved CycleGan Model and Region Segmentation [J]. Computer Science, 2020, 47(9): 142-149.
[8] DENG Liang, XU Geng-lin, LI Meng-jie, CHEN Zhang-jin. Fast Face Recognition Based on Deep Learning and Multiple Hash Similarity Weighting [J]. Computer Science, 2020, 47(9): 163-168.
[9] SU Chang, ZHANG Ding-quan, XIE Xian-zhong, TAN Ya. NFV Memory Resource Management in 5G Communication Network [J]. Computer Science, 2020, 47(9): 246-251.
[10] BAO Yu-xuan, LU Tian-liang, DU Yan-hui. Overview of Deepfake Video Detection Technology [J]. Computer Science, 2020, 47(9): 283-292.
[11] WANG Hui, LE Zi-chun, GONG Xuan, WU Yu-kun, ZUO Hao. Review of Link Prediction Methods Based on Feature Classification [J]. Computer Science, 2020, 47(8): 302-312.
[12] WANG Wen-dao, WANG Run-ze, WEI Xin-lei, QI Yun-liang, MA Yi-de. Automatic Recognition of ECG Based on Stacked Bidirectional LSTM [J]. Computer Science, 2020, 47(7): 118-124.
[13] LIU Yan, WEN Jing. Complex Scene Text Detection Based on Attention Mechanism [J]. Computer Science, 2020, 47(7): 135-140.
[14] ZHANG Zhi-yang, ZHANG Feng-li, TAN Qi, WANG Rui-jin. Review of Information Cascade Prediction Methods Based on Deep Learning [J]. Computer Science, 2020, 47(7): 141-153.
[15] JIANG Wen-bin, FU Zhi, PENG Jing, ZHU Jian. 4Bit-based Gradient Compression Method for Distributed Deep Learning System [J]. Computer Science, 2020, 47(7): 220-226.
Full text



[1] LEI Li-hui and WANG Jing. Parallelization of LTL Model Checking Based on Possibility Measure[J]. Computer Science, 2018, 45(4): 71 -75 .
[2] SUN Qi, JIN Yan, HE Kun and XU Ling-xuan. Hybrid Evolutionary Algorithm for Solving Mixed Capacitated General Routing Problem[J]. Computer Science, 2018, 45(4): 76 -82 .
[3] ZHANG Jia-nan and XIAO Ming-yu. Approximation Algorithm for Weighted Mixed Domination Problem[J]. Computer Science, 2018, 45(4): 83 -88 .
[4] WU Jian-hui, HUANG Zhong-xiang, LI Wu, WU Jian-hui, PENG Xin and ZHANG Sheng. Robustness Optimization of Sequence Decision in Urban Road Construction[J]. Computer Science, 2018, 45(4): 89 -93 .
[5] SHI Wen-jun, WU Ji-gang and LUO Yu-chun. Fast and Efficient Scheduling Algorithms for Mobile Cloud Offloading[J]. Computer Science, 2018, 45(4): 94 -99 .
[6] ZHOU Yan-ping and YE Qiao-lin. L1-norm Distance Based Least Squares Twin Support Vector Machine[J]. Computer Science, 2018, 45(4): 100 -105 .
[7] LIU Bo-yi, TANG Xiang-yan and CHENG Jie-ren. Recognition Method for Corn Borer Based on Templates Matching in Muliple Growth Periods[J]. Computer Science, 2018, 45(4): 106 -111 .
[8] GENG Hai-jun, SHI Xin-gang, WANG Zhi-liang, YIN Xia and YIN Shao-ping. Energy-efficient Intra-domain Routing Algorithm Based on Directed Acyclic Graph[J]. Computer Science, 2018, 45(4): 112 -116 .
[9] CUI Qiong, LI Jian-hua, WANG Hong and NAN Ming-li. Resilience Analysis Model of Networked Command Information System Based on Node Repairability[J]. Computer Science, 2018, 45(4): 117 -121 .
[10] WANG Zhen-chao, HOU Huan-huan and LIAN Rui. Path Optimization Scheme for Restraining Degree of Disorder in CMT[J]. Computer Science, 2018, 45(4): 122 -125 .