计算机科学

计算机科学 ›› 2026, Vol. 53 ›› Issue (8): 165-173.doi: 10.11896/jsjkx.250700148

• 计算机图形学 & 多媒体 • 上一篇    下一篇

联合跟踪的特征聚合:遮挡场景下的视频目标检测任务

王家辉, 王洪玉, 郝应光   

  1. 大连理工大学信息与通信工程学院 辽宁 大连 116024
  • 收稿日期:2025-07-22 修回日期:2025-11-26 出版日期:2026-08-15 发布日期:2026-08-17
  • 通讯作者: 王洪玉(whyu@dlut.edu.cn)
  • 作者简介:(wjhdut1008@163.com)
  • 基金资助:
    大连市科技创新基金(2022JJ11CG002)

Feature Aggregation with Joint Tracking:Video Object Detection in Occlusion Scenarios

WANG Jiahui, WANG Hongyu, HAO Yingguang   

  1. School of Information and Communication, Dalian University of Technology, Dalian, Liaoning 116024, China
  • Received:2025-07-22 Revised:2025-11-26 Published:2026-08-15 Online:2026-08-17
  • About author:WANG Jiahui,born in 2001,postgra-duate.His main research interest is vi-deo object detection.
    WANG Hongyu,born in 1968,Ph.D,professor.His main research interests include object detection,object tracking and video image processing.
  • Supported by:
    Dalian Science and Technology Innovation Fund(2022JJ11CG002).

PDF (PC)

115

摘要： 视频目标检测(VID)的主要任务是利用视频帧之间的时序信息来辅助检测。与图像目标检测不同,视频目标检测能够利用丰富的时序上下文信息,通过参考其他帧的特征来增强当前帧的检测效果。因此,现有方法普遍致力于聚合多帧间的特征,以构建时序增强的特征表示,从而提升检测的准确性。然而,在视频序列中,目标常因遮挡、运动模糊等因素出现外观退化,导致单帧检测器难以生成高质量的候选区域,使得后续的跨帧特征聚合过程失效。为此,提出一种联合跟踪与检测的视频目标检测方法,旨在改善候选区域缺失导致的聚合失败问题。具体而言,该方法首先引入轻量级跟踪器,并设计特征选择模块,以协同利用检测信息与跟踪器提供的运动先验;其次,采用加权框融合(WBF)算法对检测框与跟踪预测框进行合并,以生成在遮挡等挑战场景下更为可靠的候选区域;最后,提取这些高质量候选区域及其对应轨迹上的特征,送入后续模块进行时序聚合。在ImageNet VID数据集上的实验结果表明,所提方法有效增强了模型在遮挡场景下的鲁棒性,展现出更高的检测精度。

关键词: 视频目标检测, 目标检测, 目标跟踪, 特征聚合, 加权框融合

Abstract: Video object detection(VID) aims to exploit temporal information between video frames to assist object detection.Unlike still-image detection,VID can leverage rich temporal context by referencing features from other frames to enhance the detection performance.Therefore,existing methods generally focus on aggregating features across multiple frames to construct temporally enhanced feature representations and improve detection accuracy.However,in video sequences,objects often undergo appearance degradation due to factors such as occlusion and motion blur,making it difficult for single-frame detectors to generate high-quality candidate regions.This,in turn,can cause the subsequent cross-frame feature aggregation to fail.To address this issue,this paper proposes a tracking-assisted video object detection method designed to mitigate aggregation failures caused by missing candidate regions.Specifically,it introduces a lightweight tracker and designs a feature selection module to jointly utilize detection outputs and motion priors provided by the tracker.It then employs the WBF(Weighted Box Fusion) algorithm to merge detection boxes with tracking predictions,producing more reliable candidate regions under challenging conditions such as occlusion.Finally,it extracts these high-quality candidate regions and their corresponding trajectory features for subsequent temporal aggregation.Experiments on the ImageNet VID dataset demonstrate that the proposed method effectively enhances robustness in occluded scenarios and achieves higher detection accuracy compared with previous VID approaches.

Key words: Video object detection, Object detection, Object tracking, Feature aggregation, Weighted box fusion

中图分类号: 

  • TP391
[1] ZOU Z,CHEN K,SHI Z,et al.Object detection in 20 years:A survey[J].Proceedings of the IEEE,2023,111(3):257-276.
[2] ZHU X,DAI J,YUAN L,et al.Towards high performance videoobject detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:7210-7218.
[3] QI Q,WANG X,HOU T,et al.FastVOD-Net:A real-time and high-accuracy video object detector[J].IEEE Transactions on Intelligent Transportation Systems,2022,23(11):20926-20942.
[4] KANG K,LI H,XIAO T,et al.Object detection in videos with tubelet proposal networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:727-735.
[5] GUO M H,XU T X,LIU J,et al.Attention mechanisms in computer vision:A survey[J].Computational Visual Media,2022,8(3):331-368.
[6] HU J,SHEN L,SUN G.Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2018:7132-7141.
[7] WOO S,PARK J,LEE J Y,et al.CBAM:Convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision(ECCV).Springer,2018:3-19.
[8] ZHU X,SU W,LU L,et al.Deformable detr:Deformable transformers for end-to-end object detection[J].arXiv:2010.04159,2020.
[9] GIRSHICK R.Fast R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision.2015:1440-1448.
[10] ZHU X,XIONG Y,DAI J,et al.Flow-guided feature aggregation for video object detection[C]//Proceedings of the IEEE International Conference on Computer Vision(ICCV).IEEE,2017:408-417.
[11] SUN D,ROTH S,BLACK M J.Secrets of optical flow estimation and their principles[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.IEEE,2010:2432-2439.
[12] CHEN K,SUN X,LI J,et al.Memory enhanced global-local aggregation for video object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2020:10326-10335.
[13] GONG T,WANG C,ZHANG Q,et al.TROIA:A temporal region-based object interaction aware approach for video object detection[C]//Proceedings of the AAAI Conference on Artificial Intelligence.AAAI,2021:1424-1432.
[14] HE K,GKIOXARI G,DOLLÁR P,et al.Mask R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision(ICCV).IEEE,2017:2961-2969.
[15] HE F,GAO N,JIA J,et al.QueryProp:Object Query Propagation for High-Performance Video Object Detection[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2022:834-842.
[16] WU H,CHEN Y,WANG N,et al.Sequence level semantics aggregation for video object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2021:3026-3035.
[17] SHI W,RAJKUMAR R.Point-GNN:Graph neural network for 3d object detection in a point cloud[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:1711-1719.
[18] ZHOU Q,LI X,HE L,et al.TransVOD:End-to-end video object detection with spatial-temporal transformers[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2023,45(6):7853-7869.
[19] DUTSON M,LI Y,GUPTA M.Eventful transformers:Leveraging temporal redundancy in vision transformers[C]//Procee-dings of the IEEE/CVF International Conference on Computer Vision.2023:16911-16923.
[20] ANWAR N,BILODEAU G A,BOUACHIR W.STF:Spatio-Temporal Fusion Module for Improving Video Object Detection[J].arXiv:2402.10752,2024.
[21] ROH S D,CHUNG K S.DiffusionVID:Denoising Object Boxes With Spatio-Temporal Conditioning for Video Object Detection[J].IEEE Access,2023,11:121434-121444.
[22] BEWLEY A,GE Z,OTT L,et al.Simple online and real time tracking[C]//Proceedings of the IEEE International Conference on Image Processing(ICIP).IEEE,2016:3464-3468.
[23] ZHANG Y,WANG C,WANG X,et al.ByteTrack:Multi-Object Tracking by Associating Every Detection[C]//Proceedings of the European Conference on Computer Vision(ECCV).Sprin-ger,2022:1-17.
[24] ZHOU X,WANG D,KRÄHENBÜHL P.CenterTrack:Trac-king objects as points[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2021:9974-9983.
[25] ZHANG Y,WANG C,WANG X,et al.FairMOT:On the fairness of detection and re-identification in multiple object tracking[J].International Journal of Computer Vision,2022,130(2):306-317.
[26] FEICHTENHOFER C,PINZ A,ZISSERMAN A.Detect totrack and track to detect[C]//Proceedings of the IEEE International Conference on Computer Vision(ICCV).IEEE,2017:3038-3046.
[27] LYU Y,YANG M Y,VOSSELMAN G,et al.Plug & Play Convolutional Regression Tracker for Video Object Detection[J].arXiv:2003.00981,2020.
[28] HAN L,YIN Z.Global memory and local continuity for video object detection[J].IEEE Transactions on Multimedia,2022,25:3681-3693.
[29] SHI Y,WANG N,GUO X.YOLOV:Making still image object detectors great at video object detection[C]//Proceedings of the AAAI Conference on Artificial Intelligence(AAAI).2023:2254-2262.
[30] HASHMI K A,SHEIKH T U,STRICKER D,et al.Beyondboxes:Mask-guided spatio-temporal feature aggregation for vi-deo object detection[C]//Proceedings of the Winter Conference on Applications of Computer Vision(WACV).IEEE,2025:8111-8122.
[31] REN S,HE K,GIRSHICK R,et al.Faster R-CNN:Towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(6):1137-1149.
[32] ZHOU Z,CUI Y,ZHANG J,et al.Weighted boxesfusion:En-sembling boxes for object detection models[J].arXiv:1910.13302,2019.
[33] RUSSAKOVSKY O,DENG J,SU H,et al.ImageNet large scale visual recognition challenge[J].International Journal of Computer Vision,2015,115(3):211-252.
[34] ZHU H,WEI H,LI B,et al.A review of video object detection:Datasets,metrics and methods[J].Applied Sciences,2020,10(21):7834.
[35] QI J,GAO Y,HU Y,et al.Occluded video instance segmentation:A benchmark[J].International Journal of Computer Vision,2022,130(8):2022-2039.
[36] DENG C,CHEN D,WU Q.Identity-consistent aggregation for video object detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision(ICCV).2023.
[37] ZHANG B,WANG S,LIU Y,et al.Object detection difficulty:Suppressing over-aggregation for faster and better video object detection[C]//Proceedings of the 31st ACM International Conference on Multimedia.2023:1768-1778.
[38] SUN G,HUA Y,HU G,et al.Efficient one-stage video object detection by exploiting temporal consistency[C]//Proceedings of the European Conference on Computer Vision(ECCV).Springer,2022.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市两江新区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发