多目标跟踪中的数据关联技术综述

doi:10.11896/jsjkx.200200041

计算机科学 ›› 2020, Vol. 47 ›› Issue (10): 136-144.doi: 10.11896/jsjkx.200200041

• 计算机图形学&多媒体 • 上一篇下一篇

多目标跟踪中的数据关联技术综述

龚轩¹, 乐孜纯², 王慧¹, 武玉坤¹

1 浙江工业大学计算机科学与技术学院杭州310023
2 浙江工业大学理学院杭州310023

收稿日期:2020-02-08 修回日期:2020-07-05 出版日期:2020-10-15 发布日期:2020-10-16
通讯作者: 乐孜纯(lzc@zjut.edu.cn)
作者简介:8545919@qq.com
基金资助:
浙江省国际科技合作“一带一路”专项(2015C04005)

Survey of Data Association Technology in Multi-target Tracking

GONG Xuan¹, LE Zi-chun², WNAG Hui¹, WU Yu-kun¹

1 College of Computer Science and Technology,Zhejiang University of Technology,Hangzhou 310023,China
2 College of Science,Zhejiang University of Technology,Hangzhou 310023,China

Received:2020-02-08 Revised:2020-07-05 Online:2020-10-15 Published:2020-10-16
About author:GONG Xuan,born in 1973,Ph.D,is a member of China Computer Federation.His main research interests include AI,machine learning,deep learning and computational vision.
LE Zi-chun,born in 1965,professor,Ph.D supervisor.Her main research interests include photoelectric detection technology and instruments,optical communication.
Supported by:
“the Belt and Road” International Cooperation of Zhejiang Province (2015C04005)

摘要/Abstract

摘要： 目标跟踪一直都是计算视觉领域研究的热点课题之一,作为计算视觉的基础学科,其应用已经渗透到各个领域,包括智能监控、智能人机交互、无人驾驶以及军事等方面。目标跟踪从跟踪对象的数量角度可分为单目标跟踪和多目标跟踪,其中单目标跟踪相对简单,除了需要解决与多目标跟踪共性的问题(如遮挡、形变等)外,单目标跟踪不需要考虑目标的数据关联问题。然而,在多目标跟踪系统中,场景更为复杂,跟踪目标的数量和类别往往是不确定的,因此数据关联在整个跟踪系统中就显得尤为重要。数据关联是多目标跟踪过程中的一个重要阶段,国内外很多学者甚至将多目标跟踪问题看成数据关联问题,试图从数据关联过程中寻求多目标跟踪研究方法。文中重点对多目标跟踪过程中的数据关联技术进行了综述,系统地介绍了多目标跟踪中的数据关联技术。首先,对目标跟踪,尤其是多目标跟踪进行了概述,并对数据关联的研究现状做了描述;其次,详细介绍了数据关联的概念及其需要解决的问题;然后,对各种数据关联技术进行了分析总结,包括传统的NNDA算法、JPDA算法、基于Tracking-By-Detecting 的多目标跟踪框架的数据关联技术以及多目标多相机跟踪(Multi-Target Multi-Camera Tracking,MTMCT)的数据关联;最后,对未来多目标跟踪的数据关联技术的研究方向进行了展望。

关键词: MTMCT, Tracking-By-Detecting, 单目标跟踪, 多目标多相机跟踪, 多目标跟踪, 数据关联

Abstract: Target tracking has always been one of the hot topics in the field of computer vision.As the fundamental science of computer vision,it is applied to various fields,including intelligent monitoring,intelligent human-computer interaction,unpiloted driving and military.Target tracking can be divided into single-target tracking and multi-target tracking from the perspective of the number of tracking objects,single-target tracking is relatively simple and does not need to consider the data association of targets besides the common problems(e.g.occlusion,deformation etc.) with multi-target tracking.In the multi-target tracking system,the scenes are more complex,the number and category of targets are often uncertain,so the data association is particularly important.Data association is an important stage in the process of multi-target tracking.Many scholars domestic and abroad even regard the multi-target tracking as the problem of data association,trying to seek the research method of multi-target tracking from the process of data association.In this paper,the data association technology of multi-target tracking is reviewed and introduced systematically.Firstly,this paper gives an overview of target tracking,especially multi-target tracking,and describes the status of data association research.Secondly,the concept of data association and the problems to be solved are described in detail.Then,all kinds of data association technology are analyzed and summarized,including traditional NNDA algorithm,JPDA algorithm,data association based on the Tracking-By-Detecting framework and data association based on MTMCT(Multi Target,Multi-Camera Tracking,MTMCT).Finally,the research direction of data association technology for multi-target tracking in the future is prospected.

Key words: Data association, MTMCT, Multi-target Multi-camera tracking, Multi-target tracking, Single-target tracking, Tracking-By-Detecting

中图分类号:

TP391

龚轩, 乐孜纯, 王慧, 武玉坤. 多目标跟踪中的数据关联技术综述[J]. 计算机科学, 2020, 47(10): 136-144. https://doi.org/10.11896/jsjkx.200200041

GONG Xuan, LE Zi-chun, WNAG Hui, WU Yu-kun. Survey of Data Association Technology in Multi-target Tracking[J]. Computer Science, 2020, 47(10): 136-144. https://doi.org/10.11896/jsjkx.200200041

参考文献

[1]MENG L,YANG X.A Survey of Object Tracking Algorithms[J].Acat Automatica Sinica,2019,45(7):1244-260.
[2]WANG X.Intelligent multi-camera video surveillance:A review[J].Pattern Recognit.Lett.,2013,34(1):3-19.
[3]PFISTER T,CHARLES J,ZISSERMAN A.Flowing convnets for human pose estimation in videos[C]//Proceedings of the IEEE International Conference on Computer Vision(ICCV).Piscataway,NJ:IEEE Press,2015:1913-1921.
[4]CHOI W,SAVARESE S.A unified framework for multi-target tracking and collective activity recognition[C]//European Conference on Computer Vision.Switzerland:Springer,Cham,2012:215-230.
[5]HU W,TAN T,WANG L,et al.A survey on visual surveillanceof object motion and behaviors[J].IEEE Trans.Syst.Man Cybern.Part C-Appl.Rev.,2004,34(3):334-352.
[6]XU H,YU H W,DONG M C,et al.Overview of UAV Object Tracking[J].Journal of Network New Media,2019,8(5):11-20.
[7]YANG H X,SHAO L,ZHENG F,et al.Recent advances and trends in visual tracking:a review[J].Neurocom-puting,2011,74(18):3823-3831.
[8]SMEULDERS A W M,CHU D M,CUCCHIARA R,et al.Visual tracking:an experimental survey[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2014,36(7):1442-1468.
[9]HENRIQUES J F,RUI C,MARTINS P,et al.Exploiting thecirculant structure of tracking-by—detection with kernels[C]//European Conference on Computer Vision.Switzerland:Springer,Cham,2012:702-715.
[10]HENRIQUES J F,CASEIRO R,et al.High-Speed Trackingwith Kernelized Correlation Filters[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2014,37(3):583 - 596.
[11]MARTIN D,GUSTAV H,FAHAD K,et al.Accurate Scale Estimation for Robust Visual Tracking[C]// British Machine Vision Conference 2014.Nottingham,UK:BMVA Press,2014.
[12]MARTIN D,GUSTAV H,FAHAD K,et al.DiscriminativeScale Space Tracking[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2016,39(8):1561-1575.
[13]BOLME D S,BEVERIDGE J R,DRAPER B,et a1.Visual object tracking using adaptive correlation filters[C]// 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Press,2010:2544-2550.
[14]ALEX B,GE Z Y,LIONEL O,et al.Simple online and realtime tracking[C]//2016 IEEE International Conference on Image Processing.Piscataway,NJ:IEEE Press,2016:3464-3468.
[15]NICOLAI W,ALEX B,DIETRICH P.Simple online and real-time tracking with a deep association metric[C]//2017 IEEE International Conference on Image Processing (ICIP).Piscataway,NJ:IEEE Press,2017:3645-3649.
[16]KIM C,LI F,CIPTADI A,et al.Multiple Hypothesis Tracking Revisited[C]// 2015 IEEE International Conference on Computer Vision (ICCV).Piscataway,NJ:IEEE Press,2015:4696-4704.
[17]CHU Q,YANG W O,LI H,et al.Online Multi-Object Tracking Using CNN-based Single Object Tracker with Spatial-Temporal Attention Mechanism[C]//2017 IEEE International Conference on Computer Vision (ICCV).Piscataway,NJ:IEEE Press,2017:4846-4855.
[18]YU F W,LI W B,LI Q Q,et al.POI:Multiple Object Tracking with High Performance Detection and Appearance Feature[C]//European Conference on Computer Vision,2016.Switzerland:Springer,Cham,2016:36-42.
[19]NIMA M,SEYED M A,MOHAMMAD R.Multi-target trac-king using cnn-based features:Cnnmtt[J].Multimedia Tools and Applications,2019,78(6):7077-7096.
[20]WAN X Y,WANG J J,ZHOU S P.An online and flexible multi-object tracking framework using long short-term memory[C]//2018 IEEE CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).Piscataway,NJ:IEEE Press,2018:1230-1238.
[21]LI M H,LIU Z X,XIONG Y Y,et al.Multi-person tracking by discriminative affinity model and hierarchical association[C]//2017 3rd IEEE International Conference on Computer and Communications (ICCC).Piscataway,NJ:IEEE Press,2017:1741-1745.
[22]LUO W H,XING J L,ANTON M,et al.Multiple Object Tracking:A Literature Review[J].ArXiv Preprint ArXiv:1409.7618,2014.
[23]HU H G,ZHOU L L,GUAN Q,et al.An automatic tracking method for multiple cells based on multi-feature fusion[J].IEEE Access,2018,6:69782-69793.
[24]FIAZ M,MAHMOOD A,JUNG S K.Tracking noisy targets:A review of recent object tracking approaches[J].arXiv:1802.03098.
[25]FIAZ M,MAHMOOD A,JAVED S,et al.Handcrafted and deep trackers:Recent visual object tracking approaches and trends[J].ACM Computing Surveys,2019,52(2):1-44.
[26]LI P,WANG D,WANG L,et al.Deep visual tracking:Review and experimental comparison[J].Pattern Recognition,2018,76(4):323-338.
[27]BOSE B,WANG X,GRIMSON E.Multi-class object trackingalgorithm that handles fragmentation and grouping[C]//2007 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Press,2007:1-8.
[28]SONG B,JEN T Y,STAUDT G E,et al.A stochastic graph evolution framework for robust multi-target tracking[C]//In Proc.Eur.Conf.Comput.Vis.,BERLIN:SPRINGER-VERLAG,2010:605-619.
[29]GIOELE C,FRANCISCO L S,SIHAM T,et al.Deep learning in video multi-object tracking:A survey[J].Neurocomputing,2020,381(2):61-88.
[30]HU W,LI X,LUO W,et al.Single and multiple object tracking using log-euclidean Riemannian subspace and block-division appearance model[J].IEEE Trans.Pattern Anal.Mach.Intel.,2012,34(12):2420-2440.
[31]ZHANG L,VAN DER M L.Structure preserving object tracking[C]//2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Piscataway,NJ:IEEE Press,2013:1838-1845.
[32]ZHANG J,PRESTI L L,SCLAROFF S.Online multi-persontracking by tracker hierarchy[C]// 2012 IEEE Ninth International Conference on Advanced Video and Signal-Based Surveillance.Piscataway,NJ:IEEE Press,2012:379-385.
[33]XIANG Y,ALAHI A,SAVERESE S.Learning to track:Online multi-object tracking by decision making[C]// 2015 IEEE International Conference on Computer Vision (ICCV).Piscataway,NJ:IEEE Press,2015:4705-4713.
[34]KUO C H,HUANG C,NEYATIA R.Multi-target tracking by on-line learned discriminative appearance models[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Press,2010:685-692.
[35]HENRIQUES J F,CASEIRO R,BATISTA J.Globally optimal solution to multi-object tracking with merged measurements[C]//International Conference on Computer Vision.Piscataway,NJ:IEEE Press,2011:2470-2477.
[36]SUGIMURA D,KITANI K M,OKABE T,et al.Using individuality to track individuals:Clustering individual trajectories in crowds using local appearance and frequency trait[C]//IEEE 12th International Conference on Computer Vision.Piscataway,NJ:IEEE Press,2009:1467-1474.
[37]CHOI W,SAVARESE S.Multiple target tracking in world coordinate with single,minimally calibrated camera[C]// ECCV’10 Proceedings of the 11th European Conference on Computer Vision.Berlin:Springer,2010:553-567.
[38]HONG S,KWAK S,HHAN B.Orderless tracking throughmodel-averaged posterior estimation[C]//2013 IEEE International Conference on Computer Vision.Piscataway,NJ:IEEE Press,2013:2296-2303.
[39]WANG M M,LI X F.Research on key technologies of multiple object tracking based on data association[D].Chengdu:University of Electronic Science and Technology of China,2017.
[40]WU J X.Research on data association algorithm of multi-target tracking[D].Xi’an:Xidian University,2013.
[41]WANG Y,CUI Y T,CEHN W.Data association algorithm of multi-target tracking[J].Digital Communication World,2019(9):263-263.
[42]LI Y F,ZHOU S R.Survey of online multi-object video tracking algorithms[J].Computing Technology and Automation,2018,37(1):73-82.
[43]YANG F D.Summary of data association methods in Multi-target tracking[J].Science& Technology Vision,2016,6(6):164-194.
[44]SINGGR R A,SEA R G,HOUSEWRIGHT K B.A new Filter for Optimal Tracking in Dense Multitarget Environment[C]//Proccedings of the Ninth Allerton Conference on Circuit and System Theory,Urbana-Champaign.USA:University of Illinois,1971:201-211.
[45]SINGER R A,STEIN J J.An Optimal Tracking Filter for Processing Sensor Data of Imprecisely Determined Origin in Surveillance Systems[C]// Proceedings of the Tenth IEEE Conference on Decision and Control.IEEE Press,1971:171-175.
[46]QUAN T F.New theory and technology of target tracking[M].Natioinal Defense Industry Press,2009:55-72.
[47]JAFFER A J,BAR SHALOM Y.On optimal tracking in multiple target environments [C]// Proceedings of the third Symposium on Non-Linear Estimation Theory and Its Applications.1972:112-117.
[48]BAR-SHALOM Y.Extension of the Probabilistic Data Association Filter in Multitarget Tracking[C]//Proceedings of the Fifth Symposium on Non-Linear Estimation.1974:16-21.
[49]WANG Z,ZHENG L,LIU Y,et al.Towards Real-Time Multi-Object Tracking[J].arXiv:1909.12605,2019.
[50]WELCH G,BISHOP G.An Introduction to the Kalman Filter [EB/OL].http://www.cs.unc.edu/~tracker/ref/s2001/kalman/index.html.
[51]NUMMIARO K,KOLLER-MEIER E,VAN GOOL L.Anadaptive color-based particle filter[J].Image and Vision Computing,2003,21(1):99-110.
[52]KUHN H W.The hungarian method for the assignment problem[J].Naval Research Logistics Quarterly,1955,2(1/2):83-97.
[53]RAN N,KONG L T,WANG Y H,et al.A robust multi-athlete tracking algorithm by exploiting discriminant features and long-term dependencies[C]//International Conference on Multimedia Modeling.Switzerland:Springer,Cham,2019:411-423.
[54]CHEN L T,PENG X J,REN M W.Recurrent metric networks and batch multiple hypothesis for multi-object tracking[J].IEEE Access,2019,7:3093-3105.
[55]XIANG J,ZHANG G S,HOU J H.Online multi-object tracking based on feature representation and bayesian filtering within a deep learning architecture[J].IEEE Access,2019,7:27923-27935.
[56]KWANGJIN Y,DU Y K,YOUNG-CHUL Y,et al.Data association for multi-object tracking via deep neural networks[J].Sensors,2019,19(3):559.
[57]ZHENG Z Z,YANG X D,YU Z D,et al.Joint Discriminative and Generative Learning for Person Re-identification[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Piscataway,NJ:IEEE Press,2019:2133-2142.
[58]RISTANI E,TOMASI C.Features for multi-target multi-camera tracking and re-identification[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Press,2018:6036-6046.
[59]ZHANG Z M,WU J N,ZHANG X,et al.Multi-target,multi-camera tracking by hierarchical clustering:Recent progress on dukemtmc project[J].arXiv preprint arXiv:1712.09531,2017.
[60]ERGYS R,FRANCESCO S,ROGER Z,et al.Performancemeasures and a data set for multi-target,multi-camera tracking[C]//European Conference on Computer Vision.Switzerland:Springer,Cham,2016:17-35.
[61]TESFAYE Y T,ZEMENE E,PRATI A,et al.Multi-targettracking in multiple non-overlapping cameras using fast-constrained dominant sets[J].International Journal of Computer Vision,2019,127(9):1303-1320.
[62]HOU Y,ZHENG L,WANG Z,et al.Locality Aware Appearance Metric for Multi-Target Multi-Camera Tracking[J].arXiv:1911.12037,2019.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

多目标跟踪中的数据关联技术综述

Survey of Data Association Technology in Multi-target Tracking

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

Metrics

本文评价

推荐阅读 0

[1]	沈祥培, 丁彦蕊. 多检测器融合的深度相关滤波视频多目标跟踪算法 Multi-detector Fusion-based Depth Correlation Filtering Video Multi-target Tracking Algorithm 计算机科学, 2022, 49(8): 184-190. https://doi.org/10.11896/jsjkx.210600004
[2]	文成宇, 房卫东, 陈伟. 多目标跟踪的对象初始化综述 Object Initialization in Multiple Object Tracking:A Review 计算机科学, 2022, 49(3): 152-162. https://doi.org/10.11896/jsjkx.210200048
[3]	刘彦, 秦品乐, 曾建朝. 基于YOLOv3与分层数据关联的多目标跟踪算法 Multi-object Tracking Algorithm Based on YOLOv3 and Hierarchical Data Association 计算机科学, 2021, 48(11A): 370-375. https://doi.org/10.11896/jsjkx.201000115
[4]	胡海根, 周莉莉, 周乾伟, 陈胜勇, 张俊康. 基于CNN的相衬显微图像序列的癌细胞多目标跟踪 Multi-target Tracking of Cancer Cells under Phase Contrast Microscopic Images Based on Convolutional Neural Network 计算机科学, 2019, 46(5): 279-285. https://doi.org/10.11896/j.issn.1002-137X.2019.05.043
[5]	王正宁, 周阳, 吕侠, 曾凡伟, 张翔, 张锋军. 一种基于2D和3D联合信息的改进MDP跟踪算法 Improved MDP Tracking Method by Combining 2D and 3D Information 计算机科学, 2019, 46(3): 97-102. https://doi.org/10.11896/j.issn.1002-137X.2019.03.013
[6]	赵广辉, 卓松, 徐晓龙. 基于卡尔曼滤波的多目标跟踪方法 Multi-object Tracking Algorithm Based on Kalman Filter 计算机科学, 2018, 45(8): 253-257. https://doi.org/10.11896/j.issn.1002-137X.2018.08.045
[7]	袁大龙,纪庆革. 协同运动状态估计的多目标跟踪算法 Multiple Object Tracking Algorithm via Collaborative Motion Status Estimation 计算机科学, 2017, 44(Z11): 154-159. https://doi.org/10.11896/j.issn.1002-137X.2017.11A.032
[8]	闫林,阮宁,闫硕,高伟. 相关集的数据关联描述及实例讨论 Description of Data Association Occurring in Related Sets and Specific Example 计算机科学, 2017, 44(1): 283-288. https://doi.org/10.11896/j.issn.1002-137X.2017.01.052
[9]	陈金广,秦晓姗,马丽丽. 快速多目标跟踪GM-PHD滤波算法 Fast GM-PHD Filter for Multi-target Tracking 计算机科学, 2016, 43(3): 317-321. https://doi.org/10.11896/j.issn.1002-137X.2016.03.060
[10]	杨国亮,张进辉. 分层关联的多目标跟踪算法研究 Research on Multi-object Tracking Using Hierarchical Data Association 计算机科学, 2014, 41(9): 306-310. https://doi.org/10.11896/j.issn.1002-137X.2014.09.059
[11]	马炳先,崔纪鹏,张正明. 面向组合的Web服务间数据关联定位方法研究 Research on Data Association Location Method for Web Services Composition 计算机科学, 2014, 41(8): 130-134. https://doi.org/10.11896/j.issn.1002-137X.2014.08.029
[12]	金鑫,梁雪春,袁晓龙. 复杂情况下的多目标跟踪统计技术 Multi-target Tracking Statistical Techniques in Complex Case 计算机科学, 2013, 40(6): 268-271.
[13]	朱晓钢,杨兵,许华杰. 支持无线传感器网络多目标跟踪的聚类数据关联算法研究 Clustering Data Association Algorithm to Support Multi-target Tracl}ing in WSN 计算机科学, 2012, 39(Z6): 24-27.
[14]	周维,许海霞,郑金华. 基于RJMCMC的视觉多目标跟踪算法 Multi-object Visual Tracking Based on Reversible Jump Markov Chain Monte Carlo 计算机科学, 2012, 39(7): 270-275.
[15]	朱晓钢,杨兵,许华杰. 支持无线传感器网络多目标跟踪的最邻近数据关联算法研究 Nearest Neighbor Method Data Association Algorithm to Support Multi-target Tracking in WSN 计算机科学, 2011, 38(5): 67-70.