基于双域调谐门控与参数高效迁移学习的高光谱目标跟踪

doi:10.11896/jsjkx.260300152

摘要/Abstract

摘要： 针对高光谱目标跟踪中训练样本不足、跨模态差异显著以及光谱表征与空间建模协同不足等问题,提出一种基于双域调谐门控与参数高效迁移学习的高光谱目标跟踪方法。首先,构建面向高光谱目标跟踪任务的跨模态协同调谐机制,实现空间结构一致性约束下的光谱-空间联合建模,从而减少跨模态信息映射过程中的结构信息损失并提升匹配可靠性。其次,设计参数高效迁移学习适配结构,在冻结预训练主干参数条件下实现跨模态特征分布自适应调节,以少量可训练参数完成高光谱跟踪任务的高效适配,并缓解小样本条件下全量微调易引发的过拟合与知识退化问题。最后,在HOTC20与HOTC24数据集上的实验结果表明,所提方法在跟踪精度、成功率和参数效率方面均优于多数对比方法,在HOTC20上取得0.930的精确率和0.693的成功率,在更具挑战性的HOTC24数据集上同样表现出较强的鲁棒性与竞争力。

关键词: 高光谱目标跟踪, 参数高效迁移学习, 光谱-空间特征, 门控融合

Abstract: To address the problems of insufficient training samples,significant cross-modal discrepancies,and inadequate coordination between spectral representation and spatial modeling in hyperspectral object tracking,a hyperspectral object tracking me-thod based on the dual-domain tuning and parameter-efficient transfer learning is proposed.Firstly,dual-domain harmony gate(DDHG) module is designed to perform dynamic interaction and adaptive fusion of RGB and hyperspectral features in the two-dimensional feature space,thereby enhancing the consistency and complementarity of cross-modal information.Then,flexible normalization adapter(FNA) is introduced to achieve efficient adaptation to the hyperspectral tracking task with only a small number of trainable parameters while keeping the pretrained backbone frozen,thus alleviating the overfitting and knowledge degradation caused by full fine-tuning under limited-sample conditions.Finally,experimental results on the HOTC20 and HOTC24 datasets show that the proposed method outperforms most competing methods in terms of tracking precision,success rate,and parameter efficiency.Specifically,it achieves a precision score of 0.930 and a success score of 0.693 on HOTC20,while also demonstrating strong robustness and competitiveness on the more challenging HOTC24 dataset.

Key words: Hyperspectral object tracking, Parameter-efficient transfer learning, Spectral-spatial feature, Gate fusion

中图分类号:

TP751.1

赵徐增, 郑天托, 崔放, 刘兆国, 卢树华. 基于双域调谐门控与参数高效迁移学习的高光谱目标跟踪[J]. 计算机科学, 2026, 53(10): 174-188. https://doi.org/10.11896/jsjkx.260300152

ZHAO Xuzeng, ZHENG Tiantuo, CUI Fang, LIU Zhaoguo, LU Shuhua. Hyperspectral Object Tracking via Dual-domain Tuning Gate and Parameter-efficient TransferLearning[J]. Computer Science, 2026, 53(10): 174-188. https://doi.org/10.11896/jsjkx.260300152

参考文献

[1] XIONG F C,ZHOU J,QIAN Y T.Material based object trac-king in hyperspectral videos[J].IEEE Transactions on Image Processing,2020,29:3719-3733.
[2] WANG Q,CHEN X K,MENG H Y,et al.UAV hyperspectral data combined with machine learning for winter wheat canopy spad values estimation[J].Remote Sensing,2023,15(19):4658.
[3] STUARTM B,MCGONIGLE A J S,WILLMOTT J R.Hyperspectral imaging in environmental monitoring:a review of recent developments and technological advances in compact field deployable systems[J].Sensors,2019,19(14):3071.
[4] UZKENT B,RANGNEKAR A,HOFFMAN M J.Tracking inaerial hyperspectral videos using deep kernelized correlation filters[J].IEEE Transactions on Geoscience and Remote Sensing,2019,57(1):449-461.
[5] ZHANG Z,QIAN K,DU J,et al.Multi-features integrationbased hyperspectral videos tracker[C]//2021 11th Workshop on Hyperspectral Imaging and Signal Processing:Evolution in Remote Sensing(WHISPERS).New York:IEEE,2021:1-5.
[6] CHEN L L,ZHAO Y Q,YAO J X,et al.Object tracking in hyperspectral-oriented video with fast spatial-spectral features[J].Remote Sensing,2021,13(10):1-28.
[7] CHEN Y Z,YUAN Q Q,TANG Y Q,et al.SPIRIT:spectralawareness interaction network with dynamic template for hyperspectral object tracking[J].IEEE Transactions on Geoscience and Remote Sensing,2024,62:1-16.
[8] LI W,HOU Z F,ZHOU J,et al.SiamBAG:band attentiongrouping-based siamese object tracking network for hyperspectral videos[J].IEEE Transactions on Geoscience and Remote Sensing,2023,61:1-12.
[9] CHEN Y Z,YUAN Q Q,TANG Y Q,et al.SSTtrack:a unified hyperspectral video tracking framework via modeling spectral-spatial-temporal conditions[J].Information Fusion,2025,114:102658.
[10] CHEN L L,ZHAO Y Q,CHAN J C W,et al.Histograms of oriented mosaic gradients for snapshot spectral image description[J].ISPRS Journal of Photogrammetry and Remote Sensing,2022,183:79-93.
[11] CHEN L L,ZHAO Y Q,KONG S G.SFA-guided mosaic transformer for tracking small objects in snapshot spectral imaging[J].ISPRS Journal of Photogrammetry and Remote Sensing,2023,204:223-236.
[12] LI Z F,XIONG F C,ZHOU J,et al.BAE-net:a band attention aware ensemble network for hyperspectral object tracking[C]//2020 IEEE International Conference on Image Processing(ICIP).New York:IEEE,2020:2106-2110.
[13] LI Z F,YE X H,XIONG F C,et al.Spectral-spatial-temporal attention network for hyperspectral tracking[C]//2021 11th Workshop on Hyperspectral Imaging and Signal Processing:Evolution in Remote Sensing(WHISPERS).IEEE,2021:1-5.
[14] LI Z F,XIONG F C,ZHOU J,et al.Learning a deep ensemble network with band importance for hyperspectral object tracking[J].IEEE Transactions on Image Processing,2023,32:2901-2914.
[15] SUN C,WANG X Y,LIU Z Q,et al.SiamOHOT:a lightweight dual siamese network for onboard hyperspectral object tracking via joint spatial-spectral knowledge distillation[J].IEEE Tran-sactions on Geoscience and Remote Sensing,2023,61:1-12.
[16] WU Y N,JIAO L C,LIU X,et al.Domain adaptation-awaretransformer for hyperspectral object tracking[J].IEEE Transactions on Circuits and Systems for Video Technology,2024,34(9):8041-8052.
[17] LIU Z Q,WANG X Y,SHU M,et al.An anchor-free siamesetarget tracking network for hyperspectral video[C]//2021 11th Workshop on Hyperspectral Imaging and Signal Processing:Evolution in Remote Sensing(WHISPERS).IEEE,2021:1-5.
[18] LIU Z Q,WANG X Y,ZHONG Y F,et al.SiamHYPER:learning a hyperspectral object tracker from an rgb-based tracker[J].IEEE Transactions on Image Processing,2022,31:7116-7129.
[19] ZHU J W,LAI S M,CHEN X,et al.Visual prompt multi-modal tracking[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).New York:IEEE,2023:9516-9526.
[20] LI Z F,XIONG F C,LU J F,et al.Multi-domain universal representation learning for hyperspectral object tracking[J].Pattern Recognition,2025,162:111389.
[21] HOULSBY N,GIURGIU A,JASTRZEBSKI S,et al.Parameter-efficient transfer learning for NLP[C]//Proceedings of the 36th International Conference on Machine Learning(ICML).New York:PMLR,2019:2790-2799.
[22] WANG X W,ZHOU H,ZHANG M H,et al.Hyperspectralimage classification based on Swin Transformer and 3D residual multilayer fusion network[J].Computer Science,2023,50(5):155-160.
[23] VAN NGUYEN H,BANERJEE A,CHELLAPPA R.Tracking via object reflectance using a hyperspectral video camera[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops.IEEE,2010:44-51.
[24] QIAN K,ZHOU J,XIONG F C,et al.Object tracking in hyperspectral videos with convolutional features and kernelized correlation filter[M]//Smart Multimedia.Cham:Springer,2018:308-319.
[25] XIONG F,SUN Y,ZHOU J,et al.Spatial-spectral-temporalcorrelation filter for hyperspectral object tracking[J].IEEE Transactions on Geoscience and Remote Sensing,2025,63:1-13.
[26] BERTINETTO L,VALMADRE J,HENRIQUES J F,et al.Fully-convolutional siamese networks for object tracking[M]//Computer Vision－ECCV 2016 Workshops.Cham:Springer,2016:850-865.
[27] LI B,WU W,WANG Q,et al.SiamRPN++:evolution of siamese visual tracking with very deep networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).New York:IEEE,2019:4277-4286.
[28] VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[C]//Proceedings of the 31st International Confe-rence on Neural Information Processing Systems.2017:5998-6008.
[29] ZHAO D,ZHANG H R,HUANG K P,et al.SASU-net:hyperspectral video tracker based on spectral adaptive aggregation weighting and scale updating[J].Expert Systems with Applications,2025,272:126721.
[30] WANG Y,MEI S H,MA M Y,et al.Hyperspectral objecttracking with context-aware learning and category consistency[J].IEEE Transactions on Geoscience and Remote Sensing,2025,63:1-16.
[31] DING N,QIN Y J,YANG G,et al.Parameter-efficient fine-tu-ning of large-scale pre-trained language models[J].Nature Machine Intelligence,2023,5(3):220-235.
[32] WANG J H,LIU F,JIAO L C,et al.Satellite video object trac-king based on location prompts[J].IEEE Transactions on Circuits and Systems for Video Technology,2024,34(7):6253-6264.
[33] JIA M L,TANG L M,CHEN B C,et al.Visual prompt tuning[M]//Computer Vision－ECCV 2022.Cham:Springer,2022:709-727.
[34] WANG H,LIU F,JIAO L C,et al.ViLT-clip:video and lan-guage tuning clip with multimodal prompt learning and scenario-guided optimization[J].Proceedings of the AAAI Conference on Artificial Intelligence,2024,38(6):5390-5400.
[35] YAO R,ZHANG L,ZHOU Y,et al.Hyperspectral object trac-king with dual-stream prompt[J].IEEE Transactions on Geoscience and Remote Sensing,2025,63:1-12.
[36] CHEN Y Z,TANG Y Q,SU X,et al.PHTrack:prompting forhyperspectral video tracking[J].IEEE Transactions on Geo-science and Remote Sensing,2024,62:1-18.
[37] LI M L,YE P,ZHANG L,et al.Adapter-x:a general parameter-efficient fine-tuning framework for 2d and 3d vision[J].Neurocomputing,2026,677:133113.
[38] CHEN H,TAO R,ZHANG H,et al.Conv-adapter:exploringparameter efficient transfer learning for convnets[C]//2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops(CVPRW).IEEE,2024:1551-1561.
[39] CHEN S F,GE C J,TONG Z,et al.AdaptFormer:adapting vision transformers for scalable visual recognition[C]//36th Conference on Neural Information Processing Systems(NeurIPS 2022).Red Hook:Curran Associates Inc.,2022:16664-16678.
[40] GAO L,ZHANG Y H,CHEN L K,et al.Hyperspectral adapter for object tracking based on hyperspectral video[J].arXiv:2503.22199,2025.
[41] HU E J,SHEN Y,WALLIS P,et al.LoRA:Low-rank adapta-tion of large language models[C]//10th International Confe-rence on Learning Representations(ICLR).2022.
[42] WAN H,XIE S X,LUO T,et al.SPA-tracker:spectral prompt adaptation for hyperspectral object tracking[C]//2024 14th Workshop on Hyperspectral Imaging and Signal Processing:Evolution in Remote Sensing(WHISPERS).IEEE,2024:1-5.
[43] CHEN Y Z,YUAN Q Q,TANG Y Q,et al.SENSE:hyperspectral video object tracker via fusing material and motion cues[J].Information Fusion,2024,109:102395.
[44] WANG Y,LI Y S,WANG G,et al.Multi-scale attention net-work for single image super-resolution[C]//2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops(CVPRW).IEEE,2024:5950-5960.
[45] LI D,LI L,CHEN Z Z,et al.ShiftwiseConv:small convolutional kernel with large kernel effect[C]//2025 IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition(CVPR).IEEE,2025:25281-25291.
[46] ZHU J C,CHEN X L,HE K M,et al.Transformers withoutnormalizatio[C]//Proccedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2025:14901-14911.
[47] YE B T,CHANG H,MA B P,et al.Joint feature learning and relation modeling for tracking:a one-stream framework[M]//Computer Vision－ECCV 2022.Cham:Springer,2022:341-357.
[48] WANG S Q,QIAN K,SHEN J L,et al.AD-siamrpn:anti-deformation object tracking via an improved siamese region proposal network on hyperspectral videos[J].Remote Sensing,2023,15(7):1731.
[49] ZHU Z,WANG Q,LI B,et al.Distractor-aware siamese net-works for visual object tracking[M]//Computer Vision－ECCV 2018.Cham:Springer,2018:103-119.
[50] WANG S Q,QIAN K,CHEN P.BS-siamrpn:hyperspectral video tracking based on band selection and the siamese region proposal network[C]//2022 12th Workshop on Hyperspectral Imaging and Signal Processing:Evolution in Remote Sensing(WHISPERS).New York:IEEE,2022:1-8.
[51] ISLAMM A,ZHOU J,ZHANG W C,et al.Background-aware band selection for object tracking in hyperspectral videos[J].IEEE Geoscience and Remote Sensing Letters,2023,20:1-5.
[52] ISLAMM A,ZHOU J,XING W Z,et al.UBSTrack:unifiedband selection and multimodel ensemble for hyperspectral object tracking[J].IEEE Transactions on Geoscience and Remote Sensing,2025,63:1-15.
[53] YIN D S,HU L Y,LI B,et al.5%>100%:breaking perfor-mance shackles of full fine-tuning on visual recognition tasks[C]//2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2025:20071-20081.
[54] LUO G,ZHANG Y X,ZHOU Y Y,et al.Towards parameter-efficient network pruning with re-parameterized adapter[J].International Journal of Computer Vision,2026,134(4):174.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed