Computer Science ›› 2026, Vol. 53 ›› Issue (8): 165-173.doi: 10.11896/jsjkx.250700148

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Feature Aggregation with Joint Tracking:Video Object Detection in Occlusion Scenarios

WANG Jiahui, WANG Hongyu, HAO Yingguang   

  1. School of Information and Communication, Dalian University of Technology, Dalian, Liaoning 116024, China
  • Received:2025-07-22 Revised:2025-11-26 Online:2026-08-15 Published:2026-08-17
  • About author:WANG Jiahui,born in 2001,postgra-duate.His main research interest is vi-deo object detection.
    WANG Hongyu,born in 1968,Ph.D,professor.His main research interests include object detection,object tracking and video image processing.
  • Supported by:
    Dalian Science and Technology Innovation Fund(2022JJ11CG002).

Abstract: Video object detection(VID) aims to exploit temporal information between video frames to assist object detection.Unlike still-image detection,VID can leverage rich temporal context by referencing features from other frames to enhance the detection performance.Therefore,existing methods generally focus on aggregating features across multiple frames to construct temporally enhanced feature representations and improve detection accuracy.However,in video sequences,objects often undergo appearance degradation due to factors such as occlusion and motion blur,making it difficult for single-frame detectors to generate high-quality candidate regions.This,in turn,can cause the subsequent cross-frame feature aggregation to fail.To address this issue,this paper proposes a tracking-assisted video object detection method designed to mitigate aggregation failures caused by missing candidate regions.Specifically,it introduces a lightweight tracker and designs a feature selection module to jointly utilize detection outputs and motion priors provided by the tracker.It then employs the WBF(Weighted Box Fusion) algorithm to merge detection boxes with tracking predictions,producing more reliable candidate regions under challenging conditions such as occlusion.Finally,it extracts these high-quality candidate regions and their corresponding trajectory features for subsequent temporal aggregation.Experiments on the ImageNet VID dataset demonstrate that the proposed method effectively enhances robustness in occluded scenarios and achieves higher detection accuracy compared with previous VID approaches.

Key words: Video object detection, Object detection, Object tracking, Feature aggregation, Weighted box fusion

CLC Number: 

  • TP391
[1] ZOU Z,CHEN K,SHI Z,et al.Object detection in 20 years:A survey[J].Proceedings of the IEEE,2023,111(3):257-276.
[2] ZHU X,DAI J,YUAN L,et al.Towards high performance videoobject detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:7210-7218.
[3] QI Q,WANG X,HOU T,et al.FastVOD-Net:A real-time and high-accuracy video object detector[J].IEEE Transactions on Intelligent Transportation Systems,2022,23(11):20926-20942.
[4] KANG K,LI H,XIAO T,et al.Object detection in videos with tubelet proposal networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:727-735.
[5] GUO M H,XU T X,LIU J,et al.Attention mechanisms in computer vision:A survey[J].Computational Visual Media,2022,8(3):331-368.
[6] HU J,SHEN L,SUN G.Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2018:7132-7141.
[7] WOO S,PARK J,LEE J Y,et al.CBAM:Convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision(ECCV).Springer,2018:3-19.
[8] ZHU X,SU W,LU L,et al.Deformable detr:Deformable transformers for end-to-end object detection[J].arXiv:2010.04159,2020.
[9] GIRSHICK R.Fast R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision.2015:1440-1448.
[10] ZHU X,XIONG Y,DAI J,et al.Flow-guided feature aggregation for video object detection[C]//Proceedings of the IEEE International Conference on Computer Vision(ICCV).IEEE,2017:408-417.
[11] SUN D,ROTH S,BLACK M J.Secrets of optical flow estimation and their principles[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.IEEE,2010:2432-2439.
[12] CHEN K,SUN X,LI J,et al.Memory enhanced global-local aggregation for video object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2020:10326-10335.
[13] GONG T,WANG C,ZHANG Q,et al.TROIA:A temporal region-based object interaction aware approach for video object detection[C]//Proceedings of the AAAI Conference on Artificial Intelligence.AAAI,2021:1424-1432.
[14] HE K,GKIOXARI G,DOLLÁR P,et al.Mask R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision(ICCV).IEEE,2017:2961-2969.
[15] HE F,GAO N,JIA J,et al.QueryProp:Object Query Propagation for High-Performance Video Object Detection[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2022:834-842.
[16] WU H,CHEN Y,WANG N,et al.Sequence level semantics aggregation for video object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2021:3026-3035.
[17] SHI W,RAJKUMAR R.Point-GNN:Graph neural network for 3d object detection in a point cloud[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:1711-1719.
[18] ZHOU Q,LI X,HE L,et al.TransVOD:End-to-end video object detection with spatial-temporal transformers[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2023,45(6):7853-7869.
[19] DUTSON M,LI Y,GUPTA M.Eventful transformers:Leveraging temporal redundancy in vision transformers[C]//Procee-dings of the IEEE/CVF International Conference on Computer Vision.2023:16911-16923.
[20] ANWAR N,BILODEAU G A,BOUACHIR W.STF:Spatio-Temporal Fusion Module for Improving Video Object Detection[J].arXiv:2402.10752,2024.
[21] ROH S D,CHUNG K S.DiffusionVID:Denoising Object Boxes With Spatio-Temporal Conditioning for Video Object Detection[J].IEEE Access,2023,11:121434-121444.
[22] BEWLEY A,GE Z,OTT L,et al.Simple online and real time tracking[C]//Proceedings of the IEEE International Conference on Image Processing(ICIP).IEEE,2016:3464-3468.
[23] ZHANG Y,WANG C,WANG X,et al.ByteTrack:Multi-Object Tracking by Associating Every Detection[C]//Proceedings of the European Conference on Computer Vision(ECCV).Sprin-ger,2022:1-17.
[24] ZHOU X,WANG D,KRÄHENBÜHL P.CenterTrack:Trac-king objects as points[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2021:9974-9983.
[25] ZHANG Y,WANG C,WANG X,et al.FairMOT:On the fairness of detection and re-identification in multiple object tracking[J].International Journal of Computer Vision,2022,130(2):306-317.
[26] FEICHTENHOFER C,PINZ A,ZISSERMAN A.Detect totrack and track to detect[C]//Proceedings of the IEEE International Conference on Computer Vision(ICCV).IEEE,2017:3038-3046.
[27] LYU Y,YANG M Y,VOSSELMAN G,et al.Plug & Play Convolutional Regression Tracker for Video Object Detection[J].arXiv:2003.00981,2020.
[28] HAN L,YIN Z.Global memory and local continuity for video object detection[J].IEEE Transactions on Multimedia,2022,25:3681-3693.
[29] SHI Y,WANG N,GUO X.YOLOV:Making still image object detectors great at video object detection[C]//Proceedings of the AAAI Conference on Artificial Intelligence(AAAI).2023:2254-2262.
[30] HASHMI K A,SHEIKH T U,STRICKER D,et al.Beyondboxes:Mask-guided spatio-temporal feature aggregation for vi-deo object detection[C]//Proceedings of the Winter Conference on Applications of Computer Vision(WACV).IEEE,2025:8111-8122.
[31] REN S,HE K,GIRSHICK R,et al.Faster R-CNN:Towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(6):1137-1149.
[32] ZHOU Z,CUI Y,ZHANG J,et al.Weighted boxesfusion:En-sembling boxes for object detection models[J].arXiv:1910.13302,2019.
[33] RUSSAKOVSKY O,DENG J,SU H,et al.ImageNet large scale visual recognition challenge[J].International Journal of Computer Vision,2015,115(3):211-252.
[34] ZHU H,WEI H,LI B,et al.A review of video object detection:Datasets,metrics and methods[J].Applied Sciences,2020,10(21):7834.
[35] QI J,GAO Y,HU Y,et al.Occluded video instance segmentation:A benchmark[J].International Journal of Computer Vision,2022,130(8):2022-2039.
[36] DENG C,CHEN D,WU Q.Identity-consistent aggregation for video object detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision(ICCV).2023.
[37] ZHANG B,WANG S,LIU Y,et al.Object detection difficulty:Suppressing over-aggregation for faster and better video object detection[C]//Proceedings of the 31st ACM International Conference on Multimedia.2023:1768-1778.
[38] SUN G,HUA Y,HU G,et al.Efficient one-stage video object detection by exploiting temporal consistency[C]//Proceedings of the European Conference on Computer Vision(ECCV).Springer,2022.
[1] ZHANG Zhe, LIU Junjie, ZHANG Baili. Improved DETR-based Method for CAD Graphic Element Recognition [J]. Computer Science, 2026, 53(8): 139-147.
[2] SHAN Chengcheng, MEI Chun, LI Weiting, GUO Yuanyuan, QIAN Weixing, XIONG Zhi. Semantic Perception Active Learning Method for the Datum Map of Scene Matching Navigation System [J]. Computer Science, 2026, 53(6A): 250600228-8.
[3] CHEN Nuo, ZHAO Peng, HUAN Haisheng. Review of Small Object Detection Based on Deep Learning [J]. Computer Science, 2026, 53(6A): 250700022-9.
[4] ZHENG Haibin, LIN Xiuhao, HAN Ye, CHEN Jinyin, LI Beibei. Black-box Physical Adversarial Attack Against Multimodal Object Detector [J]. Computer Science, 2026, 53(6A): 250700023-10.
[5] QU Jiewu, LU Xinxi, SUN Jian, LIU Yan, GAO Ling, XU Binbin. Object Detection Method Based on Phased Training Strategy and Multi-scale Feature Fusion [J]. Computer Science, 2026, 53(6A): 250700088-7.
[6] MAO Lihong, TANG Jianjun, CHEN Tong, ZHANG Rui. Aerial Image Object Detection Model Based on Dual-domain Attention and Feature Fusion [J]. Computer Science, 2026, 53(6A): 250600036-7.
[7] ZHANG Shouyi, SHEN Qiang, GUO Yiran, WANG Hanyu. Rain and Fog Weather Object Detection Algorithm Based on Improved YOLOv8 Model [J]. Computer Science, 2026, 53(6A): 250300090-7.
[8] LIU Dai, AN Pengyu, WANG Kai. Improved YOLOv5s-based Algorithm for Emergency Situation Detection in Airport Terminals [J]. Computer Science, 2026, 53(6A): 250300174-7.
[9] DONG Ye, LIAN Xinyue, WANG Yuyang, OU Xinyu. RGB-IR Multi-modal Fusion-based Tomato Small Object Detection [J]. Computer Science, 2026, 53(6A): 250700173-8.
[10] ZHOU Wenwu, LEI Lei, XUAN Xin. Armory Equipment Detection Based on Improved YOLOv5 [J]. Computer Science, 2026, 53(6A): 250800049-6.
[11] JI Wenyu, LI Yang, WANG Jiabao, FU Ruizhi, LIU Xiaoyu, MIAO Zhuang. Review of 3D Object Detection Based on LiDAR-camera Fusion [J]. Computer Science, 2026, 53(6): 214-231.
[12] LI Peng, ZHANG Zihao, HAN Yahong. Primitive Dynamic Weighting for Multi-modal Salient Object Detection [J]. Computer Science, 2026, 53(6): 242-251.
[13] LIU Jikang, HUANG Lei, ZHANG Ke, NIE Jie, WEI Zhiqiang. Object Detection Method Based on Dynamic Feature Fusion [J]. Computer Science, 2026, 53(6): 263-269.
[14] SONG Jianhua, LIU Chun, ZHANG Yan. Lightweight Camouflaged Object Detection Model Based on Structured Knowledge Distillation [J]. Computer Science, 2026, 53(4): 299-307.
[15] ZHAO Binbei, ZHU Li, ZHAO Hongli, LI Yutong. Computer Vision Applications in Rail Transit Systems [J]. Computer Science, 2026, 53(3): 214-224.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!