计算机科学

计算机科学 ›› 2020, Vol. 47 ›› Issue (7): 84-91.doi: 10.11896/jsjkx.190900006

• 计算机图形学&多媒体 • 上一篇    下一篇

视觉图像显著性检测综述

袁野1,2,3, 和晓歌1, 朱定坤4, 王富利4, 谢浩然5, 汪俊1, 魏明强1,2,3, 郭延文3   

  1. 1 南京航空航天大学计算机科学与技术学院／人工智能学院 南京210006
    2 模式分析与机器智能工业和信息化部重点实验室 南京210016
    3 南京大学计算机软件新技术国家重点实验室 南京210023
    4 香港公开大学 香港999077
    5 香港教育大学 香港999077
  • 收稿日期:2019-09-02 出版日期:2020-07-15 发布日期:2020-07-16
  • 通讯作者: 魏明强(mqwei@nuaa.edu.cn)
  • 作者简介:yuenye@nuaa.edu.cn
  • 基金资助:
    国家自然科学基金项目 (61502137);中央高校基本科研业务费 (NJ2019010);南京大学计算机软件新技术国家重点实验室开题课题 (KFKT2018B20);香港教育大学2018-19院长研究基金之早期职业计划种子基金 (SFG-10);香港教育大学2018-19院长研究基金之跨学科研究计划基金 (FLASS/DRF/IDS-3)

Survey of Visual Image Saliency Detection

YUAN Ye1,2,3, HE Xiao-ge1, ZHU Ding-kun4, WANG Fu-lee4, XIE Hao-ran5, WANG Jun1, WEI Ming-qiang1,2,3, GUO Yan-wen3   

  1. 1 College of Computer Science and Technology/College of Artificial Intelligence,Nanjing University of Aeronautics and Astronautics,Nanjing 210006,China
    2 MIIT Key Laboratory of Pattern Analysis and Machine Intelligence,Nanjing 210016,China
    3 State Key Laboratory for Novel Software Technology,Nanjing University,Nanjing 210023,China
    4 The Open University of Hong Kong,Hong Kong 999077,China
    5 The Education University of Hong Kong,Hong Kong 999077,China
  • Received:2019-09-02 Online:2020-07-15 Published:2020-07-16
  • About author:YUAN Ye,born in 1998,postgraduate.His main research interests include computer vision and so on.
    WEI Ming-qiang,born in 1985,Ph.D,associate professor.His main research interest is computer graphics with an emphasis on smart geometry proces-sing.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China (61502137),Fundamental Research Funds for the Central Universities (NJ2019010),Grant from State Key Laboratory for Novel Software Technology at Nanjing University (KFKT2018B20),Seed Fund for Early Career Scheme of the Dean’s Research Fund 2018-19 (SFG-10) of the Education University of Hong Kong and Interdisciplinary Research Scheme of the Dean’s Research Fund 2018-19 (FLASS/DRF/IDS-3) of the Education University of Hong Kong

PDF (PC)

1048

摘要： 当今图像数据呈爆炸式增长,如何利用计算机高效地获取、处理图片信息成为领域内重要的研究课题。在人类视觉注意机制的启发下,研究人员发现将这种机制引入机器图像处理任务中可以大大提高信息提取的效率,从而更好地节省有限的计算资源。视觉图像显著性检测即利用计算机模拟人类的视觉注意机制,对图片中各部分信息的重要程度进行计算。其在图像分割、视频压缩、目标检测、图像索引等领域得到了广泛的应用,有着重要的研究价值。文中介绍了图像显著性检测算法的研究现状,首先以信息驱动来源为切入点,对显著性检测模型进行概述,之后分析了现有几种典型的显著性检测算法,并根据是否基于学习的模型将其分为基于非学习模型、基于传统机器学习模型以及基于深度学习模型3类。针对第一类,文中较为详细地对基于局部对比度和基于全局对比度的显著性检测算法进行了分类比较,指出了各自的优势与不足;针对后两类,分析了机器学习算法及深度学习在显著性检测中的应用。最后对现有的显著性检测算法进行了总结比较,对该领域研究的下一步发展方向进行了展望。

关键词: 显著区域检测, 视觉显著性检测, 深度学习, 机器学习, 视觉注意机制

Abstract: In today’s society where image data are exploding,how to use computer to efficiently acquire and process image information has become an important research topic.Under the inspiration of human visual attention mechanism,researchers have found that when this mechanism is introduced into machine image processing tasks,the efficiency of information extraction can be greatly improved,thus saving more limited computing resources.Visual image saliency detection is to use computers to simulate the human visual attention mechanism to calculate the importance of the information of each part in the image,which has been widely used in image segmentation,video compression,target detection,image indexing and other aspects,and has important research values.This paper summarizes and introduces the research situation of image saliency detection algorithms.Firstly,it takes information-driven sources as starting point to summarize the saliency detection model,and then analyzes several typical saliency detection algorithms.The models are divided into 3 categories according to whether they are based on learning models,which are based on non-learning models,based on traditional machine learning models and based on deep learning models.For the first category,the paper compares in more details the saliency detection algorithms based on local contrast and global contrast,and points out their respective advantages and disadvantages.For the latter two categories,this paper analyzes the application of machine learning algorithms and deep learning in saliency detection.Finally,this paper summarizes and compares the existing saliency detection algorithms and prospects the future development direction of the research in this aspect.

Key words: Salientregion detection, Visual saliency detection, Deep learning, Machine learning, Visual attention mechanism

中图分类号: 

  • TP391
[1] HAN J,NGAN K,LI M,et al.Unsupervised extraction of visual attention objects in color images[J].IEEE TCSV,2006,16 (1):409-416.
[2] KO B,NAM J.Object-of-interest image segmentation based on human attention and semantic region clustering[J].Jopt Soc Am,2006,23(10):409-414.
[3] HADIZADEH H,BAJIC I V.Saliency-aware video compression[J].IEEE Transactions on Image Processing,2014,23(1):19-33.
[4] LIU F,SHEN T S,LOU S L,et al.Deep network saliency detection based on global model and local optimization[J].Acta Optica Sinica,2017,37(12):1215005.
[5] ZHENG L,WANG S J,LIU Z Q,et al.Fast imageretrieval:Query pruning and early termination[J].IEEE Transactions on Mul-timedia,2015,17(5):648-659.
[6] DESIMONE R,DUNCAN J.Neural mechanisms of selectivevisual attention[J].Annual Review of Neuroscience,1995,18(1):193-222.
[7] ITTI L,KOCH C,NIEBUR.A model of saliency-based visualattention for rapid scene analysis[J].IEEE Transactions on Pat-tern Analysis and Machine Intelligence,1998,20(11):1254-1259.
[8] MA Y F,ZHANG H J.Contrast-based image attention analysisby using fuzzy growing[C]//Proceedings of the 11th ACM Con-ference on Multimedia.NewYork:ACM,2003:374-381.
[9] ACHANTA R,ESTRADA F,WILS P,et al.Salient region detection and segmentation[C]//Proceedings of the 6th Interna-tionalConference on Computer Vision Systems.Berlin:Springer-Verlag,2008:66-75.
[10] GOFERMAN S,ZELNIK-MAMOR L,TAL A.Context-aware saliency detection[C]//Proceedings of the 23rd International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2010:2376-2383.
[11] ACHANTA R,HEMAMI S,ESTRADA F,et al.Frequency-tuned salient region detection[C]//Proceedings of the 22nd Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2009:1597-1604.
[12] ZHAI Y,SHAH M.Visual attention detection in video se-quencesusing spatiotemporal cues[C]//Proceedings of the 14th ACM Conference on Multimedia.New York:ACM Press,2006:815-824.
[13] CHENG M M,ZHANG G X,MITRA N J,et al.Global contrast-based salient region detection[C]//Proceedings of the 24th International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2011:409-416.
[14] PERAZZI F,KRAHENBUL P,PRITCH Y,et al.Saliency fil-ters:Contrast based filtering for salient region detection[C] //Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition.United States:CVPR,2012:733-740.
[15] SHI K,WANG K,LU J,et al.PISA:Pixelwise Image Saliency by Aggregating Complementary Appearance Contrast Measures with Spatial Priors[C] //Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Guangzhou:IEEE Computer Society,2013:2115-2122.
[16] LIU T,SUN J,ZHENG N,et al.Learning to detect a salient ob-ject[C] //Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Xi’an:IEEE Computer Society,2007:1-8.
[17] RAHTU E,KANNALA J,SALO M,et al.Segmenting Salient Objects from Images and Videos[C]//Proceedings of European Conference on Computer Vision.Heraklion:ECCV,2010:366-379.
[18] JIANG H Z,YUAN Z J,CHENG M M,et al.Salient object detection:A discriminative regional feature integration approach[C]//Proceedings of the 26th International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2013:2083-2090.
[19] MAI L,NIU Y,LIU F.Saliency aggregation:a datadriven approach[C]//Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition.Portland,OR,USA:IEEE Computer Society,2013:1131-1138.
[20] LU H C,TONG N,ZHANG X N,et al.Co-Bootstrapping saliency[J].IEEE Transactions on Image Processing,2017,26(1):414-425.
[21] LI G,YU Y.Visual Saliency Based on Multiscale Deep Features[C] //Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Boston,MA,USA:IEEE Compu-ter Society,2015:5455-5463.
[22] ZHAO R,OUYANG W,LI H,et al.Saliency detection by multi-context deep learning[C] //Proceedings of 2015 IEEEConfe-rence on Computer Vision and Pattern Recognition.Boston,MA,USA:IEEE Computer Society,2015:1265-1274.
[23] WANG L,LU H,RUAN X,et al.Deep networks for saliency detection via local estimation and global search[C]//Procee-dings of the 28th International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2015:3183-3192.
[24] LI G,YU Y.Deep contrast learning for salient object detection[C]//Proceedings of the 29th International Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society,2016:478-487.
[25] LIU N,HAN J.Dhsnet:Deep hierarchical saliency network for salient object detection[C]//Proceedings of the IEEE Confe-rence on Computer Vision and Pattern Recognition (CVPR).LasVegas,NV:IEEE Computer Society,2016:678-686.
[26] LEE G,TAI Y W,KIM J.Deep saliency with encoded low level distance map and high level features[C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Las Vegas,NV,USA:IEEE Computer Society,2016:660-668.
[27] HOU Q,CHENG M M,HU X,et al.Deeply supervised salient object detection with short connections[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Honolulu,HI:IEEE Computer Society,2017:5300-5309.
[28] ZHANG L,DAI J,LU H C,et al.A Bi-directional MessagePassing Model for Salient Object Detections[C]//Proceedings of The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Salt Lake City,UT,USA:IEEE Computer Society,2018:1741-1750.
[29] WU J Y.Review of Bottom-up Salient Object Detection[J].Computer Science,2019,46(3):49-52.
[1] 王瑞平, 贾真, 刘畅, 陈泽威, 李天瑞. 基于DeepFM的深度兴趣因子分解机网络[J]. 计算机科学, 2021, 48(1): 226-232.
[2] 于文家, 丁世飞. 基于自注意力机制的条件生成对抗网络[J]. 计算机科学, 2021, 48(1): 241-246.
[3] 仝鑫, 王斌君, 王润正, 潘孝勤. 面向自然语言处理的深度学习对抗样本综述[J]. 计算机科学, 2021, 48(1): 258-267.
[4] 李吟, 李必信. 基于脚本预测和重组的内存泄漏测试加速技术[J]. 计算机科学, 2020, 47(9): 31-39.
[5] 丁钰, 魏浩, 潘志松, 刘鑫. 网络表示学习算法综述[J]. 计算机科学, 2020, 47(9): 52-59.
[6] 何鑫, 许娟, 金莹莹. 行为关联网络:完整的变化行为建模[J]. 计算机科学, 2020, 47(9): 123-128.
[7] 叶亚男, 迟静, 于志平, 战玉丽, 张彩明. 基于改进CycleGan模型和区域分割的表情动画合成[J]. 计算机科学, 2020, 47(9): 142-149.
[8] 邓良, 许庚林, 李梦杰, 陈章进. 基于深度学习与多哈希相似度加权实现快速人脸识别[J]. 计算机科学, 2020, 47(9): 163-168.
[9] 苏畅, 张定权, 谢显中, 谭娅. 面向5G通信网络的NFV内存资源管理方法[J]. 计算机科学, 2020, 47(9): 246-251.
[10] 暴雨轩, 芦天亮, 杜彦辉. 深度伪造视频检测技术综述[J]. 计算机科学, 2020, 47(9): 283-292.
[11] 王慧, 乐孜纯, 龚轩, 武玉坤, 左浩. 基于特征分类的链路预测方法综述[J]. 计算机科学, 2020, 47(8): 302-312.
[12] 王文刀, 王润泽, 魏鑫磊, 漆云亮, 马义德. 基于堆叠式双向LSTM的心电图自动识别算法[J]. 计算机科学, 2020, 47(7): 118-124.
[13] 刘燕, 温静. 基于注意力机制的复杂场景文本检测[J]. 计算机科学, 2020, 47(7): 135-140.
[14] 张志扬, 张凤荔, 谭琪, 王瑞锦. 基于深度学习的信息级联预测方法综述[J]. 计算机科学, 2020, 47(7): 141-153.
[15] 蒋文斌, 符智, 彭晶, 祝简. 一种基于4Bit编码的深度学习梯度压缩算法[J]. 计算机科学, 2020, 47(7): 220-226.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] 雷丽晖,王静. 可能性测度下的LTL模型检测并行化研究[J]. 计算机科学, 2018, 45(4): 71 -75 .
[2] 孙启,金燕,何琨,徐凌轩. 用于求解混合车辆路径问题的混合进化算法[J]. 计算机科学, 2018, 45(4): 76 -82 .
[3] 张佳男,肖鸣宇. 带权混合支配问题的近似算法研究[J]. 计算机科学, 2018, 45(4): 83 -88 .
[4] 伍建辉,黄中祥,李武,吴健辉,彭鑫,张生. 城市道路建设时序决策的鲁棒优化[J]. 计算机科学, 2018, 45(4): 89 -93 .
[5] 史雯隽,武继刚,罗裕春. 针对移动云计算任务迁移的快速高效调度算法[J]. 计算机科学, 2018, 45(4): 94 -99 .
[6] 周燕萍,业巧林. 基于L1-范数距离的最小二乘对支持向量机[J]. 计算机科学, 2018, 45(4): 100 -105 .
[7] 刘博艺,唐湘滟,程杰仁. 基于多生长时期模板匹配的玉米螟识别方法[J]. 计算机科学, 2018, 45(4): 106 -111 .
[8] 耿海军,施新刚,王之梁,尹霞,尹少平. 基于有向无环图的互联网域内节能路由算法[J]. 计算机科学, 2018, 45(4): 112 -116 .
[9] 崔琼,李建华,王宏,南明莉. 基于节点修复的网络化指挥信息系统弹性分析模型[J]. 计算机科学, 2018, 45(4): 117 -121 .
[10] 王振朝,侯欢欢,连蕊. 抑制CMT中乱序程度的路径优化方案[J]. 计算机科学, 2018, 45(4): 122 -125 .
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发