视觉图像显著性检测综述

doi:10.11896/jsjkx.190900006

计算机科学 ›› 2020, Vol. 47 ›› Issue (7): 84-91.doi: 10.11896/jsjkx.190900006

• 计算机图形学&多媒体 • 上一篇下一篇

视觉图像显著性检测综述

袁野^1,2,3, 和晓歌¹, 朱定坤⁴, 王富利⁴, 谢浩然⁵, 汪俊¹, 魏明强^1,2,3, 郭延文³

1 南京航空航天大学计算机科学与技术学院／人工智能学院南京210006
2 模式分析与机器智能工业和信息化部重点实验室南京210016
3 南京大学计算机软件新技术国家重点实验室南京210023
4 香港公开大学香港999077
5 香港教育大学香港999077

收稿日期:2019-09-02 出版日期:2020-07-15 发布日期:2020-07-16
通讯作者: 魏明强(mqwei@nuaa.edu.cn)
作者简介:yuenye@nuaa.edu.cn
基金资助:
国家自然科学基金项目 (61502137);中央高校基本科研业务费 (NJ2019010);南京大学计算机软件新技术国家重点实验室开题课题 (KFKT2018B20);香港教育大学2018-19院长研究基金之早期职业计划种子基金 (SFG-10);香港教育大学2018-19院长研究基金之跨学科研究计划基金 (FLASS/DRF/IDS-3)

Survey of Visual Image Saliency Detection

YUAN Ye^1,2,3, HE Xiao-ge¹, ZHU Ding-kun⁴, WANG Fu-lee⁴, XIE Hao-ran⁵, WANG Jun¹, WEI Ming-qiang^1,2,3, GUO Yan-wen³

1 College of Computer Science and Technology/College of Artificial Intelligence,Nanjing University of Aeronautics and Astronautics,Nanjing 210006,China
2 MIIT Key Laboratory of Pattern Analysis and Machine Intelligence,Nanjing 210016,China
3 State Key Laboratory for Novel Software Technology,Nanjing University,Nanjing 210023,China
4 The Open University of Hong Kong,Hong Kong 999077,China
5 The Education University of Hong Kong,Hong Kong 999077,China

Received:2019-09-02 Online:2020-07-15 Published:2020-07-16
About author:YUAN Ye,born in 1998,postgraduate.His main research interests include computer vision and so on.
WEI Ming-qiang,born in 1985,Ph.D,associate professor.His main research interest is computer graphics with an emphasis on smart geometry proces-sing.
Supported by:
This work was supported by the National Natural Science Foundation of China (61502137),Fundamental Research Funds for the Central Universities (NJ2019010),Grant from State Key Laboratory for Novel Software Technology at Nanjing University (KFKT2018B20),Seed Fund for Early Career Scheme of the Dean’s Research Fund 2018-19 (SFG-10) of the Education University of Hong Kong and Interdisciplinary Research Scheme of the Dean’s Research Fund 2018-19 (FLASS/DRF/IDS-3) of the Education University of Hong Kong

摘要/Abstract

摘要： 当今图像数据呈爆炸式增长,如何利用计算机高效地获取、处理图片信息成为领域内重要的研究课题。在人类视觉注意机制的启发下,研究人员发现将这种机制引入机器图像处理任务中可以大大提高信息提取的效率,从而更好地节省有限的计算资源。视觉图像显著性检测即利用计算机模拟人类的视觉注意机制,对图片中各部分信息的重要程度进行计算。其在图像分割、视频压缩、目标检测、图像索引等领域得到了广泛的应用,有着重要的研究价值。文中介绍了图像显著性检测算法的研究现状,首先以信息驱动来源为切入点,对显著性检测模型进行概述,之后分析了现有几种典型的显著性检测算法,并根据是否基于学习的模型将其分为基于非学习模型、基于传统机器学习模型以及基于深度学习模型3类。针对第一类,文中较为详细地对基于局部对比度和基于全局对比度的显著性检测算法进行了分类比较,指出了各自的优势与不足;针对后两类,分析了机器学习算法及深度学习在显著性检测中的应用。最后对现有的显著性检测算法进行了总结比较,对该领域研究的下一步发展方向进行了展望。

关键词: 机器学习, 深度学习, 视觉显著性检测, 视觉注意机制, 显著区域检测

Abstract: In today’s society where image data are exploding,how to use computer to efficiently acquire and process image information has become an important research topic.Under the inspiration of human visual attention mechanism,researchers have found that when this mechanism is introduced into machine image processing tasks,the efficiency of information extraction can be greatly improved,thus saving more limited computing resources.Visual image saliency detection is to use computers to simulate the human visual attention mechanism to calculate the importance of the information of each part in the image,which has been widely used in image segmentation,video compression,target detection,image indexing and other aspects,and has important research values.This paper summarizes and introduces the research situation of image saliency detection algorithms.Firstly,it takes information-driven sources as starting point to summarize the saliency detection model,and then analyzes several typical saliency detection algorithms.The models are divided into 3 categories according to whether they are based on learning models,which are based on non-learning models,based on traditional machine learning models and based on deep learning models.For the first category,the paper compares in more details the saliency detection algorithms based on local contrast and global contrast,and points out their respective advantages and disadvantages.For the latter two categories,this paper analyzes the application of machine learning algorithms and deep learning in saliency detection.Finally,this paper summarizes and compares the existing saliency detection algorithms and prospects the future development direction of the research in this aspect.

Key words: Deep learning, Machine learning, Salientregion detection, Visual attention mechanism, Visual saliency detection

中图分类号:

TP391

袁野, 和晓歌, 朱定坤, 王富利, 谢浩然, 汪俊, 魏明强, 郭延文. 视觉图像显著性检测综述[J]. 计算机科学, 2020, 47(7): 84-91. https://doi.org/10.11896/jsjkx.190900006

YUAN Ye, HE Xiao-ge, ZHU Ding-kun, WANG Fu-lee, XIE Hao-ran, WANG Jun, WEI Ming-qiang, GUO Yan-wen. Survey of Visual Image Saliency Detection[J]. Computer Science, 2020, 47(7): 84-91. https://doi.org/10.11896/jsjkx.190900006

参考文献

[1]HAN J,NGAN K,LI M,et al.Unsupervised extraction of visual attention objects in color images[J].IEEE TCSV,2006,16 (1):409-416.
[2]KO B,NAM J.Object-of-interest image segmentation based on human attention and semantic region clustering[J].Jopt Soc Am,2006,23(10):409-414.
[3]HADIZADEH H,BAJIC I V.Saliency-aware video compression[J].IEEE Transactions on Image Processing,2014,23(1):19-33.
[4]LIU F,SHEN T S,LOU S L,et al.Deep network saliency detection based on global model and local optimization[J].Acta Optica Sinica,2017,37(12):1215005.
[5]ZHENG L,WANG S J,LIU Z Q,et al.Fast imageretrieval:Query pruning and early termination[J].IEEE Transactions on Mul-timedia,2015,17(5):648-659.
[6]DESIMONE R,DUNCAN J.Neural mechanisms of selectivevisual attention[J].Annual Review of Neuroscience,1995,18(1):193-222.
[7]ITTI L,KOCH C,NIEBUR.A model of saliency-based visualattention for rapid scene analysis[J].IEEE Transactions on Pat-tern Analysis and Machine Intelligence,1998,20(11):1254-1259.
[8]MA Y F,ZHANG H J.Contrast-based image attention analysisby using fuzzy growing[C]//Proceedings of the 11th ACM Con-ference on Multimedia.NewYork:ACM,2003:374-381.
[9]ACHANTA R,ESTRADA F,WILS P,et al.Salient region detection and segmentation[C]//Proceedings of the 6th Interna-tionalConference on Computer Vision Systems.Berlin:Springer-Verlag,2008:66-75.
[10]GOFERMAN S,ZELNIK-MAMOR L,TAL A.Context-aware saliency detection[C]//Proceedings of the 23rd International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2010:2376-2383.
[11]ACHANTA R,HEMAMI S,ESTRADA F,et al.Frequency-tuned salient region detection[C]//Proceedings of the 22nd Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2009:1597-1604.
[12]ZHAI Y,SHAH M.Visual attention detection in video se-quencesusing spatiotemporal cues[C]//Proceedings of the 14th ACM Conference on Multimedia.New York:ACM Press,2006:815-824.
[13]CHENG M M,ZHANG G X,MITRA N J,et al.Global contrast-based salient region detection[C]//Proceedings of the 24th International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2011:409-416.
[14]PERAZZI F,KRAHENBUL P,PRITCH Y,et al.Saliency fil-ters:Contrast based filtering for salient region detection[C] //Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition.United States:CVPR,2012:733-740.
[15]SHI K,WANG K,LU J,et al.PISA:Pixelwise Image Saliency by Aggregating Complementary Appearance Contrast Measures with Spatial Priors[C] //Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Guangzhou:IEEE Computer Society,2013:2115-2122.
[16]LIU T,SUN J,ZHENG N,et al.Learning to detect a salient ob-ject[C] //Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Xi’an:IEEE Computer Society,2007:1-8.
[17]RAHTU E,KANNALA J,SALO M,et al.Segmenting Salient Objects from Images and Videos[C]//Proceedings of European Conference on Computer Vision.Heraklion:ECCV,2010:366-379.
[18]JIANG H Z,YUAN Z J,CHENG M M,et al.Salient object detection:A discriminative regional feature integration approach[C]//Proceedings of the 26th International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2013:2083-2090.
[19]MAI L,NIU Y,LIU F.Saliency aggregation:a datadriven approach[C]//Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition.Portland,OR,USA:IEEE Computer Society,2013:1131-1138.
[20]LU H C,TONG N,ZHANG X N,et al.Co-Bootstrapping saliency[J].IEEE Transactions on Image Processing,2017,26(1):414-425.
[21]LI G,YU Y.Visual Saliency Based on Multiscale Deep Features[C] //Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Boston,MA,USA:IEEE Compu-ter Society,2015:5455-5463.
[22]ZHAO R,OUYANG W,LI H,et al.Saliency detection by multi-context deep learning[C] //Proceedings of 2015 IEEEConfe-rence on Computer Vision and Pattern Recognition.Boston,MA,USA:IEEE Computer Society,2015:1265-1274.
[23]WANG L,LU H,RUAN X,et al.Deep networks for saliency detection via local estimation and global search[C]//Procee-dings of the 28th International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2015:3183-3192.
[24]LI G,YU Y.Deep contrast learning for salient object detection[C]//Proceedings of the 29th International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2016:478-487.
[25]LIU N,HAN J.Dhsnet:Deep hierarchical saliency network for salient object detection[C]//Proceedings of the IEEE Confe-rence on Computer Vision and Pattern Recognition (CVPR).LasVegas,NV:IEEE Computer Society,2016:678-686.
[26]LEE G,TAI Y W,KIM J.Deep saliency with encoded low level distance map and high level features[C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Las Vegas,NV,USA:IEEE Computer Society,2016:660-668.
[27]HOU Q,CHENG M M,HU X,et al.Deeply supervised salient object detection with short connections[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Honolulu,HI:IEEE Computer Society,2017:5300-5309.
[28]ZHANG L,DAI J,LU H C,et al.A Bi-directional MessagePassing Model for Salient Object Detections[C]//Proceedings of The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Salt Lake City,UT,USA:IEEE Computer Society,2018:1741-1750.
[29]WU J Y.Review of Bottom-up Salient Object Detection[J].Computer Science,2019,46(3):49-52.

相关文章 15

[1]	饶志双, 贾真, 张凡, 李天瑞. 基于Key-Value关联记忆网络的知识图谱问答方法 Key-Value Relational Memory Networks for Question Answering over Knowledge Graph 计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[2]	冷典典, 杜鹏, 陈建廷, 向阳. 面向自动化集装箱码头的AGV行驶时间估计 Automated Container Terminal Oriented Travel Time Estimation of AGV 计算机科学, 2022, 49(9): 208-214. https://doi.org/10.11896/jsjkx.210700028
[3]	宁晗阳, 马苗, 杨波, 刘士昌. 密码学智能化研究进展与分析 Research Progress and Analysis on Intelligent Cryptology 计算机科学, 2022, 49(9): 288-296. https://doi.org/10.11896/jsjkx.220300053
[4]	汤凌韬, 王迪, 张鲁飞, 刘盛云. 基于安全多方计算和差分隐私的联邦学习方案 Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy 计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[5]	徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺. 时序知识图谱表示学习 Temporal Knowledge Graph Representation Learning 计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[6]	李瑶, 李涛, 李埼钒, 梁家瑞, Ibegbu Nnamdi JULIAN, 陈俊杰, 郭浩. 基于多尺度的稀疏脑功能超网络构建及多特征融合分类研究 Construction and Multi-feature Fusion Classification Research Based on Multi-scale Sparse Brain Functional Hyper-network 计算机科学, 2022, 49(8): 257-266. https://doi.org/10.11896/jsjkx.210600094
[7]	王剑, 彭雨琦, 赵宇斐, 杨健. 基于深度学习的社交网络舆情信息抽取方法综述 Survey of Social Network Public Opinion Information Extraction Based on Deep Learning 计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[8]	郝志荣, 陈龙, 黄嘉成. 面向文本分类的类别区分式通用对抗攻击方法 Class Discriminative Universal Adversarial Attack for Text Classification 计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[9]	姜梦函, 李邵梅, 郑洪浩, 张建朋. 基于改进位置编码的谣言检测模型 Rumor Detection Model Based on Improved Position Embedding 计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[10]	张光华, 高天娇, 陈振国, 于乃文. 基于N-Gram静态分析技术的恶意软件分类研究 Study on Malware Classification Based on N-Gram Static Analysis Technology 计算机科学, 2022, 49(8): 336-343. https://doi.org/10.11896/jsjkx.210900203
[11]	何强, 尹震宇, 黄敏, 王兴伟, 王源田, 崔硕, 赵勇. 基于大数据的进化网络影响力分析研究综述 Survey of Influence Analysis of Evolutionary Network Based on Big Data 计算机科学, 2022, 49(8): 1-11. https://doi.org/10.11896/jsjkx.210700240
[12]	孙奇, 吉根林, 张杰. 基于非局部注意力生成对抗网络的视频异常事件检测方法 Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection 计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[13]	侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木. 中文预训练模型研究进展 Advances in Chinese Pre-training Models 计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[14]	周慧, 施皓晨, 屠要峰, 黄圣君. 基于主动采样的深度鲁棒神经网络学习 Robust Deep Neural Network Learning Based on Active Sampling 计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
[15]	苏丹宁, 曹桂涛, 王燕楠, 王宏, 任赫. 小样本雷达辐射源识别的深度学习方法综述 Survey of Deep Learning for Radar Emitter Identification Based on Small Sample 计算机科学, 2022, 49(7): 226-235. https://doi.org/10.11896/jsjkx.210600138

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

视觉图像显著性检测综述

Survey of Visual Image Saliency Detection

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

Metrics

本文评价

推荐阅读 0