导航菜单

Computer Science ›› 2020, Vol. 47 ›› Issue (7): 84-91.doi: 10.11896/jsjkx.190900006

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Survey of Visual Image Saliency Detection

YUAN Ye1,2,3, HE Xiao-ge1, ZHU Ding-kun4, WANG Fu-lee4, XIE Hao-ran5, WANG Jun1, WEI Ming-qiang1,2,3, GUO Yan-wen3   

  1. 1 College of Computer Science and Technology/College of Artificial Intelligence,Nanjing University of Aeronautics and Astronautics,Nanjing 210006,China
    2 MIIT Key Laboratory of Pattern Analysis and Machine Intelligence,Nanjing 210016,China
    3 State Key Laboratory for Novel Software Technology,Nanjing University,Nanjing 210023,China
    4 The Open University of Hong Kong,Hong Kong 999077,China
    5 The Education University of Hong Kong,Hong Kong 999077,China
  • Received:2019-09-02 Online:2020-07-15 Published:2020-07-16
  • About author:YUAN Ye,born in 1998,postgraduate.His main research interests include computer vision and so on.
    WEI Ming-qiang,born in 1985,Ph.D,associate professor.His main research interest is computer graphics with an emphasis on smart geometry proces-sing.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China (61502137),Fundamental Research Funds for the Central Universities (NJ2019010),Grant from State Key Laboratory for Novel Software Technology at Nanjing University (KFKT2018B20),Seed Fund for Early Career Scheme of the Dean’s Research Fund 2018-19 (SFG-10) of the Education University of Hong Kong and Interdisciplinary Research Scheme of the Dean’s Research Fund 2018-19 (FLASS/DRF/IDS-3) of the Education University of Hong Kong

PDF (PC)

343

Abstract: In today’s society where image data are exploding,how to use computer to efficiently acquire and process image information has become an important research topic.Under the inspiration of human visual attention mechanism,researchers have found that when this mechanism is introduced into machine image processing tasks,the efficiency of information extraction can be greatly improved,thus saving more limited computing resources.Visual image saliency detection is to use computers to simulate the human visual attention mechanism to calculate the importance of the information of each part in the image,which has been widely used in image segmentation,video compression,target detection,image indexing and other aspects,and has important research values.This paper summarizes and introduces the research situation of image saliency detection algorithms.Firstly,it takes information-driven sources as starting point to summarize the saliency detection model,and then analyzes several typical saliency detection algorithms.The models are divided into 3 categories according to whether they are based on learning models,which are based on non-learning models,based on traditional machine learning models and based on deep learning models.For the first category,the paper compares in more details the saliency detection algorithms based on local contrast and global contrast,and points out their respective advantages and disadvantages.For the latter two categories,this paper analyzes the application of machine learning algorithms and deep learning in saliency detection.Finally,this paper summarizes and compares the existing saliency detection algorithms and prospects the future development direction of the research in this aspect.

Key words: Salientregion detection, Visual saliency detection, Deep learning, Machine learning, Visual attention mechanism

CLC Number: 

  • TP391
[1] HAN J,NGAN K,LI M,et al.Unsupervised extraction of visual attention objects in color images[J].IEEE TCSV,2006,16 (1):409-416.
[2] KO B,NAM J.Object-of-interest image segmentation based on human attention and semantic region clustering[J].Jopt Soc Am,2006,23(10):409-414.
[3] HADIZADEH H,BAJIC I V.Saliency-aware video compression[J].IEEE Transactions on Image Processing,2014,23(1):19-33.
[4] LIU F,SHEN T S,LOU S L,et al.Deep network saliency detection based on global model and local optimization[J].Acta Optica Sinica,2017,37(12):1215005.
[5] ZHENG L,WANG S J,LIU Z Q,et al.Fast imageretrieval:Query pruning and early termination[J].IEEE Transactions on Mul-timedia,2015,17(5):648-659.
[6] DESIMONE R,DUNCAN J.Neural mechanisms of selectivevisual attention[J].Annual Review of Neuroscience,1995,18(1):193-222.
[7] ITTI L,KOCH C,NIEBUR.A model of saliency-based visualattention for rapid scene analysis[J].IEEE Transactions on Pat-tern Analysis and Machine Intelligence,1998,20(11):1254-1259.
[8] MA Y F,ZHANG H J.Contrast-based image attention analysisby using fuzzy growing[C]//Proceedings of the 11th ACM Con-ference on Multimedia.NewYork:ACM,2003:374-381.
[9] ACHANTA R,ESTRADA F,WILS P,et al.Salient region detection and segmentation[C]//Proceedings of the 6th Interna-tionalConference on Computer Vision Systems.Berlin:Springer-Verlag,2008:66-75.
[10] GOFERMAN S,ZELNIK-MAMOR L,TAL A.Context-aware saliency detection[C]//Proceedings of the 23rd International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2010:2376-2383.
[11] ACHANTA R,HEMAMI S,ESTRADA F,et al.Frequency-tuned salient region detection[C]//Proceedings of the 22nd Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2009:1597-1604.
[12] ZHAI Y,SHAH M.Visual attention detection in video se-quencesusing spatiotemporal cues[C]//Proceedings of the 14th ACM Conference on Multimedia.New York:ACM Press,2006:815-824.
[13] CHENG M M,ZHANG G X,MITRA N J,et al.Global contrast-based salient region detection[C]//Proceedings of the 24th International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2011:409-416.
[14] PERAZZI F,KRAHENBUL P,PRITCH Y,et al.Saliency fil-ters:Contrast based filtering for salient region detection[C] //Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition.United States:CVPR,2012:733-740.
[15] SHI K,WANG K,LU J,et al.PISA:Pixelwise Image Saliency by Aggregating Complementary Appearance Contrast Measures with Spatial Priors[C] //Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Guangzhou:IEEE Computer Society,2013:2115-2122.
[16] LIU T,SUN J,ZHENG N,et al.Learning to detect a salient ob-ject[C] //Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Xi’an:IEEE Computer Society,2007:1-8.
[17] RAHTU E,KANNALA J,SALO M,et al.Segmenting Salient Objects from Images and Videos[C]//Proceedings of European Conference on Computer Vision.Heraklion:ECCV,2010:366-379.
[18] JIANG H Z,YUAN Z J,CHENG M M,et al.Salient object detection:A discriminative regional feature integration approach[C]//Proceedings of the 26th International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2013:2083-2090.
[19] MAI L,NIU Y,LIU F.Saliency aggregation:a datadriven approach[C]//Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition.Portland,OR,USA:IEEE Computer Society,2013:1131-1138.
[20] LU H C,TONG N,ZHANG X N,et al.Co-Bootstrapping saliency[J].IEEE Transactions on Image Processing,2017,26(1):414-425.
[21] LI G,YU Y.Visual Saliency Based on Multiscale Deep Features[C] //Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Boston,MA,USA:IEEE Compu-ter Society,2015:5455-5463.
[22] ZHAO R,OUYANG W,LI H,et al.Saliency detection by multi-context deep learning[C] //Proceedings of 2015 IEEEConfe-rence on Computer Vision and Pattern Recognition.Boston,MA,USA:IEEE Computer Society,2015:1265-1274.
[23] WANG L,LU H,RUAN X,et al.Deep networks for saliency detection via local estimation and global search[C]//Procee-dings of the 28th International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2015:3183-3192.
[24] LI G,YU Y.Deep contrast learning for salient object detection[C]//Proceedings of the 29th International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2016:478-487.
[25] LIU N,HAN J.Dhsnet:Deep hierarchical saliency network for salient object detection[C]//Proceedings of the IEEE Confe-rence on Computer Vision and Pattern Recognition (CVPR).LasVegas,NV:IEEE Computer Society,2016:678-686.
[26] LEE G,TAI Y W,KIM J.Deep saliency with encoded low level distance map and high level features[C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Las Vegas,NV,USA:IEEE Computer Society,2016:660-668.
[27] HOU Q,CHENG M M,HU X,et al.Deeply supervised salient object detection with short connections[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Honolulu,HI:IEEE Computer Society,2017:5300-5309.
[28] ZHANG L,DAI J,LU H C,et al.A Bi-directional MessagePassing Model for Salient Object Detections[C]//Proceedings of The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Salt Lake City,UT,USA:IEEE Computer Society,2018:1741-1750.
[29] WU J Y.Review of Bottom-up Salient Object Detection[J].Computer Science,2019,46(3):49-52.
[1] WANG Wen-dao, WANG Run-ze, WEI Xin-lei, QI Yun-liang, MA Yi-de. Automatic Recognition of ECG Based on Stacked Bidirectional LSTM [J]. Computer Science, 2020, 47(7): 118-124.
[2] LIU Yan, WEN Jing. Complex Scene Text Detection Based on Attention Mechanism [J]. Computer Science, 2020, 47(7): 135-140.
[3] ZHANG Zhi-yang, ZHANG Feng-li, TAN Qi, WANG Rui-jin. Review of Information Cascade Prediction Methods Based on Deep Learning [J]. Computer Science, 2020, 47(7): 141-153.
[4] JIANG Wen-bin, FU Zhi, PENG Jing, ZHU Jian. 4Bit-based Gradient Compression Method for Distributed Deep Learning System [J]. Computer Science, 2020, 47(7): 220-226.
[5] CHEN Jin-yin, ZHANG Dun-Jie, LIN Xiang, XU Xiao-dong and ZHU Zi-ling. False Message Propagation Suppression Based on Influence Maximization [J]. Computer Science, 2020, 47(6A): 17-23.
[6] CHENG Zhe, BAI Qian, ZHANG Hao, WANG Shi-pu and LIANG Yu. Improving Hi-C Data Resolution with Deep Convolutional Neural Networks [J]. Computer Science, 2020, 47(6A): 70-74.
[7] HE Lei, SHAO Zhan-peng, ZHANG Jian-hua and ZHOU Xiao-long. Review of Deep Learning-based Action Recognition Algorithms [J]. Computer Science, 2020, 47(6A): 139-147.
[8] SUN Zheng and WANG Xin-yu. Application of Deep Learning in Photoacoustic Imaging [J]. Computer Science, 2020, 47(6A): 148-152.
[9] ZHANG Man, LI Jie, DING Rong-li, CHENG Hao-tian and SHEN Ji. Remote Sensing Image ObJect Detection Technology Based on Improved YOLO-V2 Algorithm [J]. Computer Science, 2020, 47(6A): 176-180.
[10] LI Lin, ZHAO Kai-yue, ZHAO Xiao-yong, WEI Shuai-qin and ZHANG Bing. Contaminated and Shielded Number Plate Recognition Based on Convolutional Neural Network [J]. Computer Science, 2020, 47(6A): 213-219.
[11] GU Wan-rong, FAN Wei-Jiang, XIE Xian-fen, ZHANG Zi-ye, MAO Yi-Jun, LIANG Zao-qing and LIN Zhen-xi. Automatic Tumor Recognition in Ultrasound Images Based on Multi-model Optimization [J]. Computer Science, 2020, 47(6A): 260-267.
[12] CHEN Jin-yin, CHENG Kai-hui and ZHENG Hai-bin. Deep Learning Based Modulation Recognition Method in Low SNR [J]. Computer Science, 2020, 47(6A): 283-288.
[13] CHEN Jin-yin, JIANG Tao and ZHENG Hai-bin. Radio Modulation Recognition Based on Signal-noise Ratio Classification [J]. Computer Science, 2020, 47(6A): 310-317.
[14] PENG Wei, HU Ning and HU Jing-Jing. Overview of Research on Image Steganalysis Algorithms [J]. Computer Science, 2020, 47(6A): 325-331.
[15] BAO Zhen-shan, GUO Jun-nan, XIE Yuan and ZHANG Wen-bo. Model for Stock Price Trend Prediction Based on LSTM and GA [J]. Computer Science, 2020, 47(6A): 467-473.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] YANG Wen-hua,XU Chang,YE Hai-bo,ZHOU Yu,HUANG Zhi-qiu. Taxonomy of Uncertainty Factors in Intelligence-oriented Cyber-physical Systems[J]. Computer Science, 2020, 47(3): 11 -18 .
[2] JI Cheng-yu,ZHU Xue-feng. Study on Optimization of Design Pattern Combination Operation[J]. Computer Science, 2020, 47(3): 19 -24 .
[3] . [J]. Computer Science, 2020, 47(5): 2 .
[4] KONG Fang, LI Qi-zhi, LI Shuai. Survey on Online Influence Maximization[J]. Computer Science, 2020, 47(5): 7 -13 .
[5] WANG Hui-yan, XU Jing-wei, XU Chang. Survey on Runtime Input Validation for Context-aware Adaptive Software[J]. Computer Science, 2020, 47(6): 1 -7 .
[6] LI Ling, LI Huang-hua, WANG Sheng-yuan. Experiment on Formal Verification Process of Parser of CompCert Compiler in Trusted Compiler Design[J]. Computer Science, 2020, 47(6): 8 -15 .
[7] ZHAO Song-hui, REN Zhi-lei, JIANG He. Multi-objective Optimization Methods for Software Upgradeability Problem[J]. Computer Science, 2020, 47(6): 16 -23 .
[8] CUI Kai, ZHAO Guo-liang, ZHOU Kuan-jiu, LI Ming-chu. Model of Embedded Software for Solving Concurrent Defects[J]. Computer Science, 2020, 47(6): 24 -31 .
[9] XU Zi-xi, MAO Xin-jun, YANG Yi, LU Yao. Modeling and Simulation of Q&A Community and Its Incentive Mechanism[J]. Computer Science, 2020, 47(6): 32 -37 .
[10] HUANG Yong-tao, YAN Hua. Scene Graph Generation Model Combining Attention Mechanism and Feature Fusion[J]. Computer Science, 2020, 47(6): 133 -137 .
渝ICP备16013121号-4
Copyright © 2019 Editorial Office of Computer Science
Add:18# Honghu West Rd., Yubei District,Chongqing,China    Zip Code: 401121
Editorial Office: 023-63500828    E-mail: jsjkx12@163.com    jsjkx_bianwu@163.com(Finance)