Computer Science ›› 2023, Vol. 50 ›› Issue (6A): 220300237-9.doi: 10.11896/jsjkx.220300237

• Image Processing & Multimedia Technology • Previous Articles     Next Articles

Attentional Feature Fusion Approach for Siamese Network Based Object Tracking

LUO Huilan, LONG Jun, LIANG Miaomiao   

  1. School of Information Engineering,Jiangxi University of Science and Technology,Ganzhou,Jiangxi 341000,China
  • Online:2023-06-10 Published:2023-06-12
  • About author:LUO Huilan,born in 1974,Ph.D,professor.Her main research interests include pattern recognition,action recognition,object attracking and image classification.
  • Supported by:
    National Natural Science Foundation of China(61862031,61901198),Jiangxi Province Major Discipline Academic and Technical Leaders Training Program-Leading Talent Project(20213BCJ22004),Qingjiang Youth Talent Support Program of Jiangxi University of Technology(JXUSTQJYX2020019) and Key Project of Provincial Degree and Graduate Education Teaching Reform Research Project(JXYJG-2020-120).

Abstract: In order to solve the problem of tracking drift due to target occlusion and tracking failure due to background interfe-rence during target tracking,this paper proposes a siamese network-based object tracking method with multi-feature integration,where feature fusion and attention mechanism are introduced to build multiple region-proposal-network based tracking modules.Firstly,two adjacent residual block are squeeze-and-excitation and then effectively fused,as a way to strengthen the feature information.Secondly,the parallel convolution attention module is used to filter the interference information contained in the channel information and spatial information.Finally,an algorithm similar to ensemble learning is proposed by constructing two different trackers,which receive deep semantic features and the aforementioned fused features,respectively,and weight them and train for the final object tracking.In addition,to verify the effectiveness of the algorithm,this paper also investigates the effects of diverse fusion schemes,different training weights to each tracker and the combination ways of the modules in the proposed model.Experi-mental results on the VOT2016 and VOT2018 datasets show that the proposed multi-feature integration method can effectively improve the robustness of the object tracking compared with other siamese network-based object tracking algorithms,while ensuring high accuracy.

Key words: Object tracking, Convolutional neural networks, Siamese networks, Feature fusion, Attention mechanisms

CLC Number: 

  • TP391.4
[1]LUO H L,ZHANG W S,ZHONG R,et al.Fast compressive tracking algorithm based on motion vector and assigning weight value[J].Journal of Central South University(Science and Technology),2017,48(2):395-404.
[2]LI J C,ZHANG X X,WANG K.Moving Target TrackingMethod Based on Improved Local Binary Patterns Algorithm and Camshift[J].Science Technology and Engineering,2021,21(26):11232-11239.
[3]ZHU Z F,YANG H B,YE Y D.Adaptive Weighted ObjectTracking Algorithm Based on Multi-appearance Models[J].Pattern Recognition and Artificial Intelligence,2016,29(11):1019-1027.
[4]CHEN Y,HUI Y,HU X H.Background aware Correlation Filter Tracking Algorithm with Adaptive Scaling and Learning Rate Adjustment[J].Computer Science,2021,48(5):178-183.
[5]BERTINETTO L,VALMADRE J,HENRIQUES J F,et al.Fully-Convolutional Siamese Networks for Object Tracking[C]//14th European Conference on Computer Vision.2016:850-865.
[6]ZHANG Z,PENG H.Deeper and Wider Siamese Networks for Real-Time Visual Tracking[C]//2019 Conference on Computer Vision and Pattern Recognition.2019:4586-4595.
[7]ZHAO Y,YU Z B,LI Y C.Cross-attention Guided Siamese Network Object Tracking Algorithm[J].Computer Science,2022,49(3):163-169.
[8]LI B,YAN J,WU W,et al.High Performance Visual Tracking with Siamese Region Proposal Network[C]//2018 Conference on Computer Vision and Pattern Recognition.2018:8971-8980.
[9]DASARI M M,GORTHI R K S S.IOU-Siamtrack:IOU Guided Siamese Network for Visual Object Tracking[C]//2020 IEEE International Conference on Image Processing.2020:2061-2065.
[10]HU J,SHEN L,ALBANIE S,et al.Squeeze-and-Excitation Networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,42(8):2011-2023.
[11]WOO S,PARK J,LEE J Y,et al.CBAM:Convolutional Block Attention Module[C]//15th European Conference on Computer Vision.2018:3-19.
[12]NAIR V,HINTON G E.Rectified Linear Units Improve Re-stricted Boltzmann Machines[C]//27th International Confe-rence on Machine Learning.2010:807-814.
[13]ZAGORUYKO S,KOMODAKIS N.Learning to CompareImage Patches via Convolutional Neural Networks[C]//IEEE Conference on Computer Vision and Pattern Recognition.2015:4353-4361.
[14]KENTER T,BORISOV A,DE RIJKE M.Siamese CBOW:Optimizing Word Embeddings for Sentence Representations[C]//54th Annual Meeting of the Association for Computational Linguistics.2016:941-951.
[15]REIMERS N,GUREVYCH I.Sentence-BERT:Sentence Em-beddings using Siamese BERT-Networks[C]//9th International Joint Conference on Natural Language Processing.2019:3982-3992.
[16]REN S,HE K,GIRSHICK R,et al.Faster R-CNN:Towards Real-Time Object Detection with Region Proposal Networks[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2017,39(6):1137-1150.
[17]ZHANG D W,ZHENG Z.High Performance Visual Tracking with Siamese Actor-Critic Network[C]//2020 IEEE International Conference on Image Processing.2020:2116-2121.
[18]HAN X W,WANG Y W,XIE Y H,et al.Multi-channel scaleadaptive target tracking based on double correlation filter[J].Chinese Journal of Scientific Instrument,2019,40(11):76-84.
[19]FAN Y,SONG X N.Siamese Progressive Attention-Guided Fusion Network for Object Tracking[J].Journal of Computer-Aided Design & Computer Graphics,2021,33(2):199-206.
[20]TANG Z Y,WU X J,ZHU X F,Object Tracking with Multi-spatial Resolutionsand Adaptive Feature Fusion Based on Correlation Filters[J].Pattern Recognition and Artificial Intelligence,2020,33(1):66-74.
[21]WANG Q,ZHANG L,BERTINETTO L,et al.Fast Online Object Tracking and Segmentation:A Unifying Approach[C]//32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:1328-1338.
[22]LI B,WU W,WANG Q,et al.SiamRPN++:Evolution of Siamese Visual Tracking With Very Deep Networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:4282-4291.
[23]RUSSAKOVSKY O,DENG J,SU H,et al.ImageNet LargeScale Visual Recognition Challenge[J].International Journal of Computer Vision,2015,115(3):211-253.
[24]LIN T Y,MAIRE M,BELONGIE S,et al.Microsoft COCO:Common Objects in Context[C]//13th European Conference on Computer Vision.2014:740-755.
[25]XU N,YANG L,FAN Y,et al.YouTube-VOS:Sequence-to-Sequence Video Object Segmentation[C]//15th European Conference on Computer Vision.2018:603-619.
[26]KRISTAN M,LEONARDIS A,MATAS J,et al.The visual object tracking VOT2016 challenge results[C]//14th European Conference on Computer Vision.2016:777-823.
[27]KRISTAN M,LEONARDIS A,MATAS J,et al.The Sixth Vi-sual Object Tracking VOT2018 Challenge Results[C]//15th European Conference on Computer Vision.2018:3-53.
[28]BAI L,ZHANG H L,WANG C.Target tracking algorithmbased on efficient attention and context awareness[J].Journal of Beijing University of Aeronautics and Astronautics,2022,48(7):11.
[29]LIU R H,ZHANG J X,JIN C X,et al.Target Tracking Based on Deformable Convolution Siamese Network[J].Control and Decision,2022,37(8):7.
[30]ZHU J A,CHEN T,CAO J T.Salient region weighted correlation filter for object tracking[J].Optics and Precision Engineering,2021,29(2):363-373.
[31]LI F,TIAN C,ZUO W,et al.SiamATL:Online Update of Siamese Tracking Network via Attentional Transfer Learning[J].IEEE Transactions on Cybernetics,2021:1-15.
[32]ZHANG W,JIAO L,LI Y,et al.Sparse Learning-Based Correlation Filter for Robust Tracking[J].IEEE Transactions on Image Processing,2020(30):878-892.
[33]SHAO J N,GE H W.Siamese Object Tracking Algorithm Combining Residual Connection and Channel Attention Mechanism[J]Journal of Computer-Aided Design & Computer Graphics,2021,33(2):260-269.
[34]CHEN Z W,ZHANG Z X,SONG J,et al.Tracking Algorithm for Siamese Network Based on Target-Aware Feature Selection[J].Acta Optica Sinica,2020,40(9):110-126.
[1] ZHOU Fengfan, LING Hefei, ZHANG Jinyuan, XIA Ziwei, SHI Yuxuan, LI Ping. Facial Physical Adversarial Example Performance Prediction Algorithm Based on Multi-modal Feature Fusion [J]. Computer Science, 2023, 50(8): 280-285.
[2] SHAN Xiaohuan, SONG Rui, LI Haihai, SONG Baoyan. Event Recommendation Method with Multi-factor Feature Fusion in EBSN [J]. Computer Science, 2023, 50(7): 60-65.
[3] WU Liuchen, ZHANG Hui, LIU Jiaxuan, ZHAO Chenyang. Defect Detection of Transmission Line Bolt Based on Region Attention Mechanism andMulti-scale Feature Fusion [J]. Computer Science, 2023, 50(6A): 220200096-7.
[4] DOU Zhi, HU Chenguang, LIANG Jingyi, ZHENG Liming, LIU Guoqi. Lightweight Target Detection Algorithm Based on Improved Yolov4-tiny [J]. Computer Science, 2023, 50(6A): 220700006-7.
[5] ZHANG Changfan, MA Yuanyuan, LIU Jianhua, HE Jing. Dual Gating-Residual Feature Fusion for Image-Text Cross-modal Retrieval [J]. Computer Science, 2023, 50(6A): 220700030-7.
[6] WANG Wei, BAI Long, MA Huanchang, LIU Yanheng. Study on Safety Warning Method of Driver’s Blind Area Based on Machine Vision [J]. Computer Science, 2023, 50(6A): 220700141-7.
[7] RUAN Wang, HAO Guosheng, WANG Xia, HU Xiaoting, YANG Zihao. Fusion Multi-feature Fuzzy Model for Target Recognition and Its Application [J]. Computer Science, 2023, 50(6A): 220100138-7.
[8] ZENG Zehua, LUO Huilan. Cross-dataset Learning Combining Multi-object Tracking and Human Pose Estimation [J]. Computer Science, 2023, 50(6A): 220400199-7.
[9] HUANG Yujiao, CHEN Mingkai, ZHENG Yuan, FAN Xinggang, XIAO Jie, LONG Haixia. Text Classification Based on Weakened Graph Convolutional Networks [J]. Computer Science, 2023, 50(6A): 220700039-5.
[10] WANG Tianran, WANG Qi, WANG Qingshan. Transfer Learning Based Cross-object Sign Language Gesture Recognition Method [J]. Computer Science, 2023, 50(6A): 220300232-5.
[11] LIU Zhe, LIANG Yudong, LI Jiaying. Adaptive Image Dehazing Algorithm Based on Dynamic Convolution Kernels [J]. Computer Science, 2023, 50(6): 200-208.
[12] JIA Tianhao, PENG Li. SSD Object Detection Algorithm with Residual Learning and Cyclic Attention [J]. Computer Science, 2023, 50(5): 170-176.
[13] WU Han, NIE Jiahao, ZHANG Zhaowei, HE Zhiwei, GAO Mingyu. Deep Learning-based Visual Multiple Object Tracking:A Review [J]. Computer Science, 2023, 50(4): 77-87.
[14] WANG Xiaofei, FAN Xueqiang, LI Zhangwei. Improving RNA Base Interactions Prediction Based on Transfer Learning and Multi-view Feature Fusion [J]. Computer Science, 2023, 50(3): 164-172.
[15] MEI Pengcheng, YANG Jibin, ZHANG Qiang, HUANG Xiang. Sound Event Joint Estimation Method Based on Three-dimension Convolution [J]. Computer Science, 2023, 50(3): 191-198.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!