计算机科学

计算机科学 ›› 2023, Vol. 50 ›› Issue (6A): 220300237-9.doi: 10.11896/jsjkx.220300237

• 图像处理&多媒体技术 • 上一篇    下一篇

注意力特征融合的孪生网络目标跟踪方法

罗会兰, 龙珺, 梁苗苗   

  1. 江西理工大学信息工程学院 江西 赣州 341000
  • 出版日期:2023-06-10 发布日期:2023-06-12
  • 通讯作者: 罗会兰(luohuilan@jxust.edu.cn)
  • 基金资助:
    国家自然科学基金(61862031,61901198);江西省主要学科学术和技术带头人培养计划-领军人才项目(20213BCJ22004);江西理工大学清江青年英才支持计划(JXUSTQJYX2020019);省级学位与研究生教育教学改革研究项目重点项目(JXYJG-2020-120)

Attentional Feature Fusion Approach for Siamese Network Based Object Tracking

LUO Huilan, LONG Jun, LIANG Miaomiao   

  1. School of Information Engineering,Jiangxi University of Science and Technology,Ganzhou,Jiangxi 341000,China
  • Online:2023-06-10 Published:2023-06-12
  • About author:LUO Huilan,born in 1974,Ph.D,professor.Her main research interests include pattern recognition,action recognition,object attracking and image classification.
  • Supported by:
    National Natural Science Foundation of China(61862031,61901198),Jiangxi Province Major Discipline Academic and Technical Leaders Training Program-Leading Talent Project(20213BCJ22004),Qingjiang Youth Talent Support Program of Jiangxi University of Technology(JXUSTQJYX2020019) and Key Project of Provincial Degree and Graduate Education Teaching Reform Research Project(JXYJG-2020-120).

PDF (PC)

209

摘要： 为了解决目标跟踪过程中由于目标遮挡导致的跟踪漂移和背景干扰导致的跟踪失败问题,文中提出了一种多特征集成的孪生网络目标跟踪方法,引入特征融合模块和注意力模块构建多个区域生成网络跟踪器,以获得高效的判别特征表示及对复杂环境的分辨能力。首先将相邻两个残差块特征压缩激励后进行有效融合,以此加强特征信息;其次利用并行卷积注意力模块对特征图中包含在通道信息和空间信息中的干扰信息进行过滤;最后设计并提出了一种与集成学习类似的算法,通过构建两个跟踪器,分别接收深层语义特征与融合特征,对其进行加权训练,得到最终的跟踪结果。除此之外,为验证算法的有效性,文中还研究了不同融合方案、不同跟踪器训练权重比和不同模块的组合方式对模型的影响。在VOT2016和VOT2018数据集上的实验结果表明,提出的多特征集成方法与其他孪生网络目标跟踪算法相比,在保证算法高准确率的同时,能够有效提升目标跟踪的鲁棒性。

关键词: 目标跟踪, 卷积神经网络, 孪生网络, 特征融合, 注意力机制

Abstract: In order to solve the problem of tracking drift due to target occlusion and tracking failure due to background interfe-rence during target tracking,this paper proposes a siamese network-based object tracking method with multi-feature integration,where feature fusion and attention mechanism are introduced to build multiple region-proposal-network based tracking modules.Firstly,two adjacent residual block are squeeze-and-excitation and then effectively fused,as a way to strengthen the feature information.Secondly,the parallel convolution attention module is used to filter the interference information contained in the channel information and spatial information.Finally,an algorithm similar to ensemble learning is proposed by constructing two different trackers,which receive deep semantic features and the aforementioned fused features,respectively,and weight them and train for the final object tracking.In addition,to verify the effectiveness of the algorithm,this paper also investigates the effects of diverse fusion schemes,different training weights to each tracker and the combination ways of the modules in the proposed model.Experi-mental results on the VOT2016 and VOT2018 datasets show that the proposed multi-feature integration method can effectively improve the robustness of the object tracking compared with other siamese network-based object tracking algorithms,while ensuring high accuracy.

Key words: Object tracking, Convolutional neural networks, Siamese networks, Feature fusion, Attention mechanisms

中图分类号: 

  • TP391.4
[1]LUO H L,ZHANG W S,ZHONG R,et al.Fast compressive tracking algorithm based on motion vector and assigning weight value[J].Journal of Central South University(Science and Technology),2017,48(2):395-404.
[2]LI J C,ZHANG X X,WANG K.Moving Target TrackingMethod Based on Improved Local Binary Patterns Algorithm and Camshift[J].Science Technology and Engineering,2021,21(26):11232-11239.
[3]ZHU Z F,YANG H B,YE Y D.Adaptive Weighted ObjectTracking Algorithm Based on Multi-appearance Models[J].Pattern Recognition and Artificial Intelligence,2016,29(11):1019-1027.
[4]CHEN Y,HUI Y,HU X H.Background aware Correlation Filter Tracking Algorithm with Adaptive Scaling and Learning Rate Adjustment[J].Computer Science,2021,48(5):178-183.
[5]BERTINETTO L,VALMADRE J,HENRIQUES J F,et al.Fully-Convolutional Siamese Networks for Object Tracking[C]//14th European Conference on Computer Vision.2016:850-865.
[6]ZHANG Z,PENG H.Deeper and Wider Siamese Networks for Real-Time Visual Tracking[C]//2019 Conference on Computer Vision and Pattern Recognition.2019:4586-4595.
[7]ZHAO Y,YU Z B,LI Y C.Cross-attention Guided Siamese Network Object Tracking Algorithm[J].Computer Science,2022,49(3):163-169.
[8]LI B,YAN J,WU W,et al.High Performance Visual Tracking with Siamese Region Proposal Network[C]//2018 Conference on Computer Vision and Pattern Recognition.2018:8971-8980.
[9]DASARI M M,GORTHI R K S S.IOU-Siamtrack:IOU Guided Siamese Network for Visual Object Tracking[C]//2020 IEEE International Conference on Image Processing.2020:2061-2065.
[10]HU J,SHEN L,ALBANIE S,et al.Squeeze-and-Excitation Networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,42(8):2011-2023.
[11]WOO S,PARK J,LEE J Y,et al.CBAM:Convolutional Block Attention Module[C]//15th European Conference on Computer Vision.2018:3-19.
[12]NAIR V,HINTON G E.Rectified Linear Units Improve Re-stricted Boltzmann Machines[C]//27th International Confe-rence on Machine Learning.2010:807-814.
[13]ZAGORUYKO S,KOMODAKIS N.Learning to CompareImage Patches via Convolutional Neural Networks[C]//IEEE Conference on Computer Vision and Pattern Recognition.2015:4353-4361.
[14]KENTER T,BORISOV A,DE RIJKE M.Siamese CBOW:Optimizing Word Embeddings for Sentence Representations[C]//54th Annual Meeting of the Association for Computational Linguistics.2016:941-951.
[15]REIMERS N,GUREVYCH I.Sentence-BERT:Sentence Em-beddings using Siamese BERT-Networks[C]//9th International Joint Conference on Natural Language Processing.2019:3982-3992.
[16]REN S,HE K,GIRSHICK R,et al.Faster R-CNN:Towards Real-Time Object Detection with Region Proposal Networks[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2017,39(6):1137-1150.
[17]ZHANG D W,ZHENG Z.High Performance Visual Tracking with Siamese Actor-Critic Network[C]//2020 IEEE International Conference on Image Processing.2020:2116-2121.
[18]HAN X W,WANG Y W,XIE Y H,et al.Multi-channel scaleadaptive target tracking based on double correlation filter[J].Chinese Journal of Scientific Instrument,2019,40(11):76-84.
[19]FAN Y,SONG X N.Siamese Progressive Attention-Guided Fusion Network for Object Tracking[J].Journal of Computer-Aided Design & Computer Graphics,2021,33(2):199-206.
[20]TANG Z Y,WU X J,ZHU X F,Object Tracking with Multi-spatial Resolutionsand Adaptive Feature Fusion Based on Correlation Filters[J].Pattern Recognition and Artificial Intelligence,2020,33(1):66-74.
[21]WANG Q,ZHANG L,BERTINETTO L,et al.Fast Online Object Tracking and Segmentation:A Unifying Approach[C]//32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:1328-1338.
[22]LI B,WU W,WANG Q,et al.SiamRPN++:Evolution of Siamese Visual Tracking With Very Deep Networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:4282-4291.
[23]RUSSAKOVSKY O,DENG J,SU H,et al.ImageNet LargeScale Visual Recognition Challenge[J].International Journal of Computer Vision,2015,115(3):211-253.
[24]LIN T Y,MAIRE M,BELONGIE S,et al.Microsoft COCO:Common Objects in Context[C]//13th European Conference on Computer Vision.2014:740-755.
[25]XU N,YANG L,FAN Y,et al.YouTube-VOS:Sequence-to-Sequence Video Object Segmentation[C]//15th European Conference on Computer Vision.2018:603-619.
[26]KRISTAN M,LEONARDIS A,MATAS J,et al.The visual object tracking VOT2016 challenge results[C]//14th European Conference on Computer Vision.2016:777-823.
[27]KRISTAN M,LEONARDIS A,MATAS J,et al.The Sixth Vi-sual Object Tracking VOT2018 Challenge Results[C]//15th European Conference on Computer Vision.2018:3-53.
[28]BAI L,ZHANG H L,WANG C.Target tracking algorithmbased on efficient attention and context awareness[J].Journal of Beijing University of Aeronautics and Astronautics,2022,48(7):11.
[29]LIU R H,ZHANG J X,JIN C X,et al.Target Tracking Based on Deformable Convolution Siamese Network[J].Control and Decision,2022,37(8):7.
[30]ZHU J A,CHEN T,CAO J T.Salient region weighted correlation filter for object tracking[J].Optics and Precision Engineering,2021,29(2):363-373.
[31]LI F,TIAN C,ZUO W,et al.SiamATL:Online Update of Siamese Tracking Network via Attentional Transfer Learning[J].IEEE Transactions on Cybernetics,2021:1-15.
[32]ZHANG W,JIAO L,LI Y,et al.Sparse Learning-Based Correlation Filter for Robust Tracking[J].IEEE Transactions on Image Processing,2020(30):878-892.
[33]SHAO J N,GE H W.Siamese Object Tracking Algorithm Combining Residual Connection and Channel Attention Mechanism[J]Journal of Computer-Aided Design & Computer Graphics,2021,33(2):260-269.
[34]CHEN Z W,ZHANG Z X,SONG J,et al.Tracking Algorithm for Siamese Network Based on Target-Aware Feature Selection[J].Acta Optica Sinica,2020,40(9):110-126.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发