计算机科学 ›› 2021, Vol. 48 ›› Issue (6A): 1-9.doi: 10.11896/jsjkx.201000044

• 图像处理&多媒体技术 • 上一篇    下一篇

目标轮廓检测技术新进展

冯芙蓉, 张兆功   

  1. 黑龙江大学计算机科学技术学院 哈尔滨150080
  • 出版日期:2021-06-10 发布日期:2021-06-17
  • 通讯作者: 张兆功(2013010@hlju.edu.cn)
  • 作者简介:crazyidog@163.com
  • 基金资助:
    黑龙江省自然科学基金(F2017024,F2017025)

Recent Advances for Object Contour Detection Technology

FENG Fu-rong, ZHANG Zhao-gong   

  1. School of Computer Science and Technology,Heilongjiang University,Harbin 150080,China
  • Online:2021-06-10 Published:2021-06-17
  • About author:FENG Fu-rong,born in 1986,postgra-duate.Her main research interests include object contour detection and object detection.
    ZHANG Zhao-gong,born in 1963,Ph.D,professor.His main research interests include bioinformatics,data mining,statistical genetics,big data,cloud computing etc.
  • Supported by:
    Natural Science Foundation of Heilongjiang Province,China(F2017024,F2017025).

摘要: 轮廓检测是计算机视觉研究领域中最基础、最重要、最具挑战的问题之一。随着近年来深度学习的发展,视觉领域的其他研究方向取得了突破,例如目标检测、实例分割,这些逐渐证明了轮廓检测与其他研究方向的密切关系,因此轮廓检测任务也受到了越来越广泛的关注。文中讨论了多个主体内容,不仅包括对现有轮廓检测算法的细致回顾,而且根据轮廓检测提取特征的特点将其分为3个阶段即低层、中层和高层来介绍,还包括对应用到的数据集、性能评估指标、模型结构和模型细节、轮廓检测的应用及结果的应用进行详细分析,对轮廓检测发展进行了深入介绍。最后,还对轮廓检测所面临的挑战和未来趋势进行了分析和预测,以期为该领域后续的研究提供新思路及参考。

关键词: 分割, 计算机视觉, 聚类, 轮廓检测, 深度学习

Abstract: Object contour detection is one of the most foundational,significant and challenging problems in the field of computer vision research.With the development of deep learning in recent years,breakthroughs have been made in other research directions in the field of vision,such as object detection and instance segmentation,which gradually prove the close relationship between contour detection and other research directions,so more and more attention has been paid in contour detection.This paper discusses several main contents,including not only a detailed review of the existing contour detection algorithms,but also three stages according to the features of contour detection and extraction:low-level,middle-level and high-level,and a detailed analysis of the applied datasets,performance evaluation indicators,model structure and model details,the application of contour detection and the application of its results,so as to make a deep understanding of the development of contour detection.Finally,the challenges and future trends of contour detection are analyzed and predicted.This paper provides new ideas and references for the follow-up research in this field.

Key words: Cluster, Computer vision, Contour detection, Deep learning, Segmentation

中图分类号: 

  • TP391
[1] MING Y,LI H,HE X,et al.Contour Completion Without Region Segmentation[J].IEEE Trans Image Process,2016,25(8):3597-3611.<br /> [2] MARTIN D R,FOWLKESC C,MALIK J,et al.Learning to detect natural image boundaries using local brightness,color,and texture cues[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2004,26(5):530-549.<br /> [3] ZIOU D,TABBONE S.EDGE Detection Techniques-An Overview[J].International Journal of Pattern Recognition & Image Analysis,1998,8(4):537-559.<br /> [4] YANG J,PRICE B,COHEN S,et al.Object Contour Detection with a Fully ConvolutionalEncoder-Decoder Network[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE,2016.<br /> [5] ROTHER C,KOLMOGOROV V,BLAKE A,et al.GrabCut:Interactive foreground extraction using iterated graph cuts[J].ACM Trans.Graph.,2004,23(3):309-314.<br /> [6] ARBELAEZ P,MAIRE M,FOWLKESC,et al.Contour Detection and Hierarchical Image Segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2011,33(5):898-916.<br /> [7] FERRARI V,FEVRIER L,JURIE F,et al.Groups of Adjacent Contour Segments for Object Detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2007,30(1):36-51.<br /> [8] GIRSHICK R,DONAHUE J,DARRELL T,et al.Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation[C]//CVPR.IEEE,2014.<br /> [9] ZHANG H,JIANG K,ZHANG Y,et al.Discriminative Feature Learning for Video Semantic Segmentation[C]//2014 International Conference on Virtual Reality and Visualization (ICVRV).IEEE Computer Society,2014.<br /> [10] WANG Y,ZHAO X,LI Y,et al.Deep Crisp Boundaries:From Boundaries to Higher-Level Tasks[J].IEEE transactions on image processing:a publication of the IEEE Signal Processing So-ciety,2019,28(3):1285.<br /> [11] BERTASIUS G,SHI J,TORRESANI L,et al.Semantic Seg-mentation with Boundary Neural Fields[J].IEEE Conference on Computer Vision and Pattern Recognition (CVPR).2016:3602-3610.<br /> [12] UIJLINGS J R R,SANDE K E A V D,GEVERST,et al.Selective Search for Object Recognition[J].International Journal of Computer Vision(IJCV),2013,104(2):154-171.<br /> [13] ZITNICK C L,DOLLAR P.Edge Boxes:Locating Object Proposals from Edges[C]//European Conference on Computer Vision.Cham:Springer,2014:391-405.<br /> [14] CHEN C,SEFF A,KORNHAUSER A,et al.DeepDriving:Learning Affordance for Direct Perception in Autonomous Driving[C]//ICCV.2015:2722-2730.<br /> [15] LIU H,YAO Y,SUNZ,et al.Road Segmentation with Image-LiDAR Data Fusion[C]//ECCV 2020.2020.<br /> [16] ZHANG M,HU X,ZHAOL,et al.Learning Dual Multi-Scale Manifold Ranking for Semantic Segmentation of High-Resolution Images[J].Remote Sensing,2017,9(9):500.<br /> [17] HARIHARAN B,ARBELÁEZ P,GIRSHICK R,et al.Simultaneous Detection and Segmentation[C]//European Conference on Computer Vision.Springer,2014:297-312.<br /> [18] HARIHARAN B,ARBELÁEZ P,GIRSHICK R,et al.Hypercolumns for Object Segmentation and Fine-grained Localization[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Pecognition(CVPR).2015:447-456.<br /> [19] DAI J,HE K,SUN J,et al.Instance-aware Semantic Segmentation via Multi-task Network Cascades[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:3150-3158.<br /> [20] KAIMING H,GEORGIA G,PIOTR D,et al.Mask R-CNN[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2017:1-1.<br /> [21] LE C Y,BEN G Y,HINTON G,et al.Deep learning[EB/OL].(2015-05-28)[2020-09-26].https://doi.org/10.1038/nature14539.<br /> [22] DAVID A,FORSYTH,JEAN P,et al.Computer Vision:AModern Approach,2/E[M]//Computer Vision:A Modern Approach.Prentice Hall Professional Technical Reference,2002.<br /> [23] CANNY J.A computational approach to edge detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1986,8(6):679-698.<br /> [24] MALIK J,BELONGIE S,LEUNG T,et al.Contour and Texture Analysis for Image Segmentation[J].International Journal of Computer Vision,2001,43(1):7-27.<br /> [25] LIM J J,ZITNICK C L,DOLLAR P,et al.Sketch Tokens:A Learned Mid-level Representation for Contour and Object Detection[C]//2013 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2013.<br /> [26] PIOTR D,ZITNICK L C.Structured Forests for Fast Edge Detection[C]//IEEE International Conference on Computer Vision(CVPR).IEEE Computer Society,2013.<br /> [27] GANIN Y,LEMPITSKY V.N<sup>4</sup>-Fields:Neural Network Nearest Neighbor Fields for Image Transforms[C]//Asian Conference on Computer Vision.Cham:Springer,2014.<br /> [28] BERTASIUS G,SHI J,TORRESANI L.DeepEdge:A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection[C]//IEEE Conference on Computer Vision and Pattern Recognition(CVPR).2015:4380-4389.<br /> [29] KRIZHEVSKY A,SUTSKEVER I,HINTON G,et al.Image-Net Classification with Deep Convolutional Neural Networks[J].Advances in Neural Information Processing Systems,2012,25(2).<br /> [30] SHEN W,WANG X,WANGY,et al.DeepContour:A deep convolutional feature learned by positive-sharing loss for contour detection[C]//Computer Vision & Pattern Recognition.IEEE,2015:3982-3991.<br /> [31] XIE S,TU Z.Holistically-Nested Edge Detection [C]//IEEEInternational Conference on Computer Vision (ICCV).2015:1395-1403.<br /> [32] EVAN S,JONATHAN L,TREVORD,et al.Fully ConvolutionalNetworks for Semantic Segmentation[M].IEEE Computer So-ciety,2017.<br /> [33] MANINIS K K,PONT-TUSET J,PABLO ARBELÁEZ,et al.Convolutional Oriented Boundaries[C]//European Conference on Computer Vision.Springer,2016:580-596.<br /> [34] UIJLINGS J,FERRARV.Situational object boundary detection[C]//CVPR.2015:4712-4721.<br /> [35] PHILIPP K,KOLTUN V.Efficient Inference in Fully ConnectedCRFs with Gaussian Edge Potentials[C]//Advances in Neural Information Processing Systems.2011.<br /> [36] LIU Y,CHENG M M,HU X,et al.Richer Convolutional Fea-tures for Edge Detection [C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2017:5872-5881.<br /> [37] WANG Y,ZHAO X,HUANG K,et al.Deep Crisp Boundaries[C]//IEEE Conference on Computer Vision and Pattern Recognition.2017:3892-3900.<br /> [38] AKBARINIA A,PARRAGA C A.Feedback and surround modu-lated boundary detection [J].International Journal of ComputerVision,2018,126(12):1367-1380.<br /> [39] KELM A P,RAO V S,ZOELZERU,et al.Object Contour and Edge Detection with RefineContourNet[J].arXiv:1904.13353,2019.<br /> [40] LIN G,MILAN A,SHENC,et al.RefineNet:Multi-path Refinement Networks for High-Resolution Semantic Segmentation[C]//CVPR.2017:5168-5177.<br /> [41] POMA X S,RIBA E,SAPPAA D,et al.Dense Extreme Inception Network:Towards a Robust CNN Model for Edge Detection[C]//WACV.2020:1912-1921.<br /> [42] SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-Scale Image Recognition[J].arXiv:1409.1556,2014.<br /> [43] SZEGEDY C,LIU W,JIA Y,et al.Going deeper with convolutions[C]//CVPR.2015:1-9.<br /> [44] HE K,ZHANG X,REN S,et al.Deep Residual Learning forImage Recognition[C]//IEEE Conference on Computer Vision &Pattern Recognition.IEEE Computer Society,2016:770-778.<br /> [45] YU F,KOLTUN V.Multi-Scale Context Aggregation by Dilated Convolutions[C]//International Conference on Learning Representations (ICLR).2016.<br /> [46] CHOLLET F.Xception:Deep Learning with Depthwise Separable Convolutions[J].arXiv:1610.02357,2016.<br /> [47] XIE E,SUN P,SONG X,et al.PolarMask:Single Shot Instance Segmentation With Polar Representation[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).IEEE,2020.<br /> [48] TIAN Z,SHEN C,CHEN H,et al.FCOS:Fully Convolutional One-Stage Object Detection[J].arXiv:1904.01355,2019.<br /> [49] LIN T Y,DOLLAR P,GIRSHICKR,et al.Feature PyramidNetworks for Object Detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2017.<br /> [50] MÉLY D A,KIM J,MCGILLM,et al.A systematic comparison between visual cues for boundary detection[J].Vision Research.2016,120:93-107.<br /> [51] GRIGORESCU C,PETKOV N,WESTENBERGM A,et al.Contour detection based on nonclassical receptive field inhibition[J].IEEE Trans Image Process,2003,12(7):729-739.<br /> [52] DAVID R M,CHARLESS F,DORON T,et al.A Database of Human Segmented Natural Images and its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics[C]//ICCV.2001:416-425.<br /> [53] SILBERMAN N,HOIEM D,KOHLI P,et al.Indoor Segmentation and Support Inference from RGBD Images[C]//Procee-dings of the 12th European conference on Computer Vision-Vo-lume Part V.Berlin:Springer,2012.<br /> [54] MOTTAGHI R,CHEN X,LIU X,et al.The Role of Context for Object Detection and Semantic Segmentation in the Wild[C]//Computer Vision & Pattern Recognition.IEEE,2014.<br /> [55] EVERINGHAM M,ESLAMI S M A,VAN GOOL L,et al.The Pascal Visual Object Classes Challenge:A Retrospective[J].International Journal of Computer Vision,2015,111(1):98-136.<br /> [56] HE J Z,ZHANG S L,YANG M,et al.Bi-Directional Cascade Network for Perceptual Edge Detection[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).IEEE,2020.<br /> [57] BERTASIUS G,SHI J,TORRESANI L,et al.High-for-Lowand Low-for-High:Efficient Boundary Detection from Deep Object Features and Its Applications to High-Level Vision[C]//ICCV.IEEE,2015.<br /> [58] HARIHARAN B,ARBELÁEZ P,GIRSHICK R,et al.Simultaneous Detection and Segmentation[J].Lecture Notes in Computer ence,2014,8695:297-312.<br /> [59] MING M C,YUN L,WENY L,et al.BING:Binarized normed gradients for objectness estimation at 300fps[J].Computational Visual Media,2019,5(1):4-21.<br /> [60] REN S,HE K,GIRSHICK R,et al.Faster R-CNN:Towards Real-Time Object Detection with Region Proposal Networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,39(6).<br /> [61] CARREIRA J,SMINCHISESCUC.Constrained parametric min-cuts for automatic object segmentation[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2011,34(7):3241-3248.<br /> [62] ARBELAEZ P,PONT T J,BARRON J,et al.Multiscale Combinatorial Grouping[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE,2014.<br /> [63] ROSS B G,JEFF D,TREVOR D,et al.Rich Feature Hierar-chies for Accurate Object Detection and Semantic Segmentation[C]//CVPR.2014:580-587.<br /> [64] HE K,ZHANG X,REN S,et al.Spatial Pyramid Pooling inDeep Convolutional Networks for Visual Recognition[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2014,37(9):1904-16.<br /> [65] GIRSHICK R.Fast R-CNN[J].Computer Science,2015.<br /> [66] JI F D,YI L,KAIM H,et al.R-FCN:Object Detection via Region-based Fully Convolutional Networks[J].arXiv:1605.06409,2016.<br /> [67] REDMON J,DIVVALA S,GIRSHICKR,et al.You Only Look Once:Unified,Real-Time Object Detection[C]//CVPR.2016:779-788.<br /> [68] LIU W,ANGUELOV D,ERHAND,et al.SSD:Single ShotMultiBox Detector[C]//European Conference on Computer Vision.Cham:Springer,2016.<br /> [69] WANG J,YAO J,ZHANG Y,et al.Collaborative Learning for Weakly Supervised Object Detection[C]//Twenty-Seventh International Joint Conference on Artificial Intelligence(IJCAI-18).2018.<br /> [70] AHN J,CHO S,KWAK S,et al.Weakly Supervised Learning ofInstance Segmentation with Inter-pixel Relations[C]//IEEE/CVF Conference on Computer Vision & Pattern Recognition.IEEE,2019.<br /> [71] B X W A,C H M,D S Y,et al.Deep clustering for weakly-supervised semantic segmentation in autonomous driving scenes[J].Neurocomputing,2020,381:20-28.<br /> [72] KHOREVA A,BENENSON R,OMRAN M,et al.Weakly Supervised Object Boundaries[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE,2016.<br /> [73] PEDRO F F,DANIELP H.Efficient graph-based image segmentation[J].IJCV,2004,59(2):167-181.<br /> [74] PONT T J,ARBELAEZ P,BARRONJ T,et al.Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation[J].arXiv:1503.00848,2015.<br /> [75] ZHANG Y T,ZHANG Z G.Research on the Correlation tween Hot News and Stock Index Volatility Based on Deep Learning.[C]//The 5th International Conference on Electronics Engineering and Informatics.ICEEI,2020.<br /> [76] ZHANG S,WEN L,BIAN X,et al.Occlusion-aware R-CNN:Detecting Pedestrians in a Crowd[C]//European Conference on Computer Vision.Cham:Springer,2018.<br /> [77] WANG X L,XIAO T T,JIANGY N,et al.Repulsion Loss:Detecting Pedestrians in a Crowd[C]//CVPR.2018:7774-7783.<br /> [78] ZHOU C L,YUAN J S.Bi-box Regression for Pedestrian Detection and Occlusion Estimation[C]//ECCV.2018:138-154.
[1] 鲁晨阳, 邓苏, 马武彬, 吴亚辉, 周浩浩.
基于分层抽样优化的面向异构客户端的联邦学习
Federated Learning Based on Stratified Sampling Optimization for Heterogeneous Clients
计算机科学, 2022, 49(9): 183-193. https://doi.org/10.11896/jsjkx.220500263
[2] 饶志双, 贾真, 张凡, 李天瑞.
基于Key-Value关联记忆网络的知识图谱问答方法
Key-Value Relational Memory Networks for Question Answering over Knowledge Graph
计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[3] 汤凌韬, 王迪, 张鲁飞, 刘盛云.
基于安全多方计算和差分隐私的联邦学习方案
Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy
计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[4] 柴慧敏, 张勇, 方敏.
基于特征相似度聚类的空中目标分群方法
Aerial Target Grouping Method Based on Feature Similarity Clustering
计算机科学, 2022, 49(9): 70-75. https://doi.org/10.11896/jsjkx.210800203
[5] 徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺.
时序知识图谱表示学习
Temporal Knowledge Graph Representation Learning
计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[6] 王剑, 彭雨琦, 赵宇斐, 杨健.
基于深度学习的社交网络舆情信息抽取方法综述
Survey of Social Network Public Opinion Information Extraction Based on Deep Learning
计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[7] 郝志荣, 陈龙, 黄嘉成.
面向文本分类的类别区分式通用对抗攻击方法
Class Discriminative Universal Adversarial Attack for Text Classification
计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[8] 姜梦函, 李邵梅, 郑洪浩, 张建朋.
基于改进位置编码的谣言检测模型
Rumor Detection Model Based on Improved Position Embedding
计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[9] 杨文坤, 原晓佩, 陈小锋, 郭睿.
三维激光雷达点云空间多特征分割
Spatial Multi-feature Segmentation of 3D Lidar Point Cloud
计算机科学, 2022, 49(8): 143-149. https://doi.org/10.11896/jsjkx.210300275
[10] 孙奇, 吉根林, 张杰.
基于非局部注意力生成对抗网络的视频异常事件检测方法
Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection
计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[11] 胡艳羽, 赵龙, 董祥军.
一种用于癌症分类的两阶段深度特征选择提取算法
Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification
计算机科学, 2022, 49(7): 73-78. https://doi.org/10.11896/jsjkx.210500092
[12] 程成, 降爱莲.
基于多路径特征提取的实时语义分割方法
Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction
计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[13] 侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木.
中文预训练模型研究进展
Advances in Chinese Pre-training Models
计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[14] 周慧, 施皓晨, 屠要峰, 黄圣君.
基于主动采样的深度鲁棒神经网络学习
Robust Deep Neural Network Learning Based on Active Sampling
计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
[15] 苏丹宁, 曹桂涛, 王燕楠, 王宏, 任赫.
小样本雷达辐射源识别的深度学习方法综述
Survey of Deep Learning for Radar Emitter Identification Based on Small Sample
计算机科学, 2022, 49(7): 226-235. https://doi.org/10.11896/jsjkx.210600138
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!