计算机科学

计算机科学 ›› 2023, Vol. 50 ›› Issue (6A): 221100046-8.doi: 10.11896/jsjkx.221100046

• 图像处理&多媒体技术 • 上一篇    下一篇

基于度量正则化的红外与可见光跨模态行人重识别算法

吴含笑1, 赵倩倩1, 朱建清2,4, 曾焕强2, 杜吉祥3, 廖昀4   

  1. 1 华侨大学信息科学与工程学院 福建 厦门 361021;
    2 华侨大学工学院 福建 泉州 362011;
    3 华侨大学计算机科学与技术学院 福建 厦门 361021;
    4 厦门亿联网络技术股份有限公司 福建 厦门 361015
  • 出版日期:2023-06-10 发布日期:2023-06-12
  • 通讯作者: 朱建清(jqzhu@hqu.edu.cn)
  • 作者简介:(jqzhu@hqu.edu.cn)
  • 基金资助:
    福建省杰出青年科学基金(2022J06023);国家自然科学基金(61976098)

Metric Regularized Infrared and Visible Cross-modal Person Re-identification

WU Hanxiao1, ZHAO Qianqian1, ZHU Jianqing2,4, ZENG Huanqiang2, DU Jixiang3, LIAO Yun4   

  1. 1 College of Information Science and Engineering,Huaqiao University,Xiamen,Fujian 361021,China;
    2 College of Engineering,Huaqiao University,Quanzhou,Fujian 362011,China;
    3 College of Computer Science and Technology,Huaqiao University,Xiamen,Fujian 361021,China;
    4 Xiamen Yealink Network Technology Co.,LTD,Xiamen,Fujian 361015,China
  • Online:2023-06-10 Published:2023-06-12
  • About author:WU Hanxiao,born in 1997,postgra-duate.Her main research interest is object re-identification. ZHU Jianqing,born in 1987,professor,master supervisor.His main research interests include deep learning,machine vision and pattern recognition.
  • Supported by:
    Natural Science Foundation for Outstanding Young Scholars of Fujian Province(2022J06023) and National Natural Science Foundation of China(61976098).

PDF (PC)

670

摘要： 红外与可见光跨模态行人重识别对提升智能视频监控系统的全天作战能力具有重要作用。现有跨模态行人重识别方法通常专注于特征对齐,未重视多模态度量对齐,导致重识别对模态变化不够鲁棒。为此,提出一种基于度量正则化的红外与可见光跨模态行人重识别算法。首先,设计度量正则化损失函数,用于约束不同模态检索模式下匹配行为的差异,提升算法的鲁棒性。其次,考虑到实际监控场景中红外图像的数量少于可见光图像的数量,利用模态数据比例修正交叉熵损失函数,减少模态数据不平衡对模型训练的不利影响。实验结果证明了所提算法的优越性,例如在RegDB数据集由可见光检索红外图像的首位识别率达到89.52%。

关键词: 度量正则化损失函数, 度量对齐, 跨模态, 行人重识别, 智能视频监控系统

Abstract: Infrared and visible cross-modal person re-identification plays an important role in improving the all-day combat capability of intelligent video surveillance systems.Existing methods usually focus on the alignment of cross-modal features,and neglect the metric alignment among multiple modalities,resulting in re-identification lacking robustness to modal changes.For that,this paper proposes a metric regularized infrared and visible cross-modal person re-identification.First,this paper designs a metric regularized loss function to constrain the difference among matching behaviors under different modal retrieval modes to improve the robustness.Second,considering that the number of infrared images is less than that of visible images in actual surveillance scenes,this paper applies the modal data proportion to modify the cross-entropy function to reduce the adverse effect of the imba-lance between different modalities.Experimental results show the superiority of the proposed method,e.g.,using visible images to retrieval infrared images,the rank-1 identification rate reaches 89.52% on the RegDB dataset.

Key words: Metric regularized loss function, Metric alignment, Cross-modal, Person re-identification, Intelligent video surveillance systems

中图分类号: 

  • TP391.41
[1]WANG X D,ZHENG Z D,HE Y,et al.Soft Person Reidentification Network Pruning via Bloc-kwise Adjacent Filter Decaying[J].IEEE Transaction on Cybernetics,2022,52(12):13293-13307.
[2]XU F R,MA B P,CHANG H,et al.PRDP:Person Re-identfication with Dirty and Poor Data[J].IEEE Transactions on Cybernetics,2022,52(10):11014-11026..
[3]ZHANG X F,SONG B.A Person Reidentification Method Basedon Improved Triple Loss and Feature Fusion[J].Journal of Computer-Aided Computer Science,2021,48(9):146-152.
[4]GAO Z,GAO L S,ZHANG H,et al.DCR:a Unified Framework for Holistic/Partial Person Reid[J].IEEE Transactions on Multimedia,2020,23:3332-3345.
[5]HUANG Y,WU Q,XU J S,et al.Alleviating Modality BiasTraining for Infrared-Visible Person Re-Identification[J].IEEE Transactions on Multimedia,2022,24:1570-1582.
[6]YE M,WANG Z,LAN X Y,et al.Visible Thermal Person Re-Identification via Dual-Constrained Top-Ranking[C]//Procee-dings of the International Joint Conference on Artificial Intelligence.Stockholm,Sweden:Morgan Kaufmann,2018:1092-1099.
[7]YE M,LAN X Y,LENG Q M,et al.Cross-Modality Person Re-identification via Modality-Aware Collaborative Ensemble Learning[J].IEEE Transactions on Image Processing,2020,29:9387-9399.
[8]CHENG Y Z,XIAO G Q,TANG X Q,et al.Two-Phase Feature Fusion Network for Visible-Infrared Person Re-Identification[C]//Proceedings of International Conference on Image Processing.Anchorage,Alaska,USA:IEEE Computer Society Press,2021:1149-1153.
[9]LIU H J,TAN X H,ZHOU X C.Parameter Sharing Exploration and Hetero-Center Triplet Loss for Visible-Thermal Person Re-Identification[J].IEEE Transactions on Multimedia,2021,23:4414-4425.
[10]LIU H,MA S,XIA D,et al.SFANet:a Spectrum-Aware Feature Augmentation Network for Visible-Infrared Person Reidentification[J].IEEE Transactions on Neural Networks and Learning Systems,2023,34(4):1958-1971.
[11]YE M,SHEN J B,LIN G J,et al.Deep Learning for Person Re-Identification:A Survey and Outlook[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2022,44(6):2872-2893.
[12]YE M,SHEN J B,CRANDALL D J,et al.Dynamic Dual-Attentive Aggregation Learning for Visible-Infrared Person Re-Identification[C]//Proceedings of European Conference on Compu-ter Vision.Vitrual:Springer,2020:229-247.
[13]YE M,LAN X Y,WANG Z,et al.Bi-Directional Center-Con-strained Top-Ranking for Visible-Thermal Person Re-Identification[J].IEEE Transactions on Information Forensics and Security,2020,15:407-419.
[14]ZHANG C,LIU H,GUO W,et al.Multi-Scale Cascading Network with Compact Feature Learning for RGB-infrared Person Re-Identification[C]//Proceedings of International Conference on Pattern Recognition.Milan,Italy:IEEE Computer Society Press,2020:8679-8686.
[15]HAO Y,WANG N N,LI J,et al.HSME:Hypersphere ManifoldEmbedding for Visible Thermal Person Re-identification[C]//Proceedings of AAAI Conference on Artificial Intelligence.Honolulu,Hawaii,USA:AAAI Press,2019:8385-8392.
[16]WEI Z Y,YANG X,WANG N N,et al.Flexible Body Partition-Based Adversarial Learning for Visible-Infrared Person Re-Identification[J].IEEE Transactions on Neural Networks and Learning Systems,2022,33(9):4676-4687.
[17]FENG Z,LAI J,XIE X.Learning Modality-Specific Representations for Visible-Infrared Person Re-Identification[J].IEEE Transactions on Image Processing,2019,29:579-590.
[18]PARK H,LEE S H,LEE J,et al.Learning by Aligning:Visible-Infrared Person Re-identification Using Cross-Modal Correspondences[C]//Proceedings of International Conference on Computer Vision.Virtual:IEEE Computer Society Press,2021:12046-12055.
[19]ZHANG L Y,DU G D,LIU F,et al.Global-Local MultipleGranularity Learning for Cross-Modality Visible-Infrared Person Reidentification[J/OL].IEEE Transactions on Neural Networks and Learning Systems(Early Access),2021.https://ieeexplore.ieee.org/document/9457243.
[20]ZHONG X,LU T Y,HUANG W X,et al.Grayscale Enhancement Colorization Network for Visible-Infrared Person Re-Identification[J].IEEE Transactions on Circuits and Systems for Video Technology,2022,32(3):1418-1430.
[21]DAI H P,XIE Q,MA Y C,et al.RGB-Infrared Person Re-identification via Image Modality Conversion[C]//Proceedings of International Conference on Pattern Recognition.Milan,Italy:IEEE Computer Society Press,2020:592-598.
[22]DAI P Y,JI R R,WANG H B,et al.Cross-Modality Person Re-identification with Generative Adversarial Training[C]//Proceedings of International Joint Conference on Artificial Intelligence.Stockholm,Sweden:Morgan Kaufmann,2018:677-683.
[23]WANG G A,ZHANG T Z,YANG Y,et al.Cross-ModalityPaired-Images Generation for RGB-Infrared Person Re-Identification[C]//Proceedings of AAAI Conference on Artificial Intelligence.New York,NY,USA:AAAI Press,2020.12144-12151.
[24]WANG G A,ZHANG T Z,CHENG J,et al.RGB-infraredCross-Modality Person Re-identification Via Joint Pixel and Feature Alignment[C]//Proceedings of International Conference on Computer Vision.Seoul,Korea:IEEE Computer Society Press,2019:3622-3631.
[25]LIU H,MIAO Z L,YANG B,et al.A Base-Derivative Framework for Cross-Modality RGB-infrared Person Re-Identification[C]//Proceedings of International Conference on Pattern Re-cognition.Milan,Italy:IEEE Computer Society Press,2020:7640-7646.
[26]HU B Y,LIU J W,ZHA Z J.Adversarial Disentanglement and Correlation Network for RGB-infrared Person Re-Identification[C]//Proceedings of International Conference on Multimedia and Expo.Shenzhen,China:IEEE Computer Society Press,2021:1-6.
[27]WEI Z Y,YANG X,WANG N N,et al.Syncretic Modality Collaborative Learning for Visible Infrared Person Re-Identification[C]//Proceedings of International Conference on Computer Vision.Virtual:IEEE Computer Society Press,2021:225-243.
[28]WU D,WANG C,WU Y,et al.Attention Deep Model withMulti-Scale Deep Supervision for Person Re-Identification[J].IEEE Transactions on Emerging Topics in Computational Intelligence,2021,5(1):70-78.
[29]ZHONG Z,ZHENG L,ZHENG Z D,et al.CamStyle:A Novel Data Augmentation Method for Person Re-Identification[J].IEEE Transactions on Image Processing,2019.28(3):1176-1190.
[30]SUN Y,WANG X,TANG X.Deep Learning Face Representation from Predicting 10,000 Classes[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Columbus,OH,USA:IEEE Computer Society Press,2014:1891-1898.
[31]HAO X,ZHAO S Y,YE M,et al.Cross-Modality Person Re-Identification via Modality Confusion and Center Aggregation[C]//Proceedings of International Conference on Computer Vision.Virtual:IEEE Computer Society Press,2021:16383-16392.
[32]HE K M,ZHANG X Y,REN S Q,et al.Deep Residual Learning for Image Recognition[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Las Vegas,NV,USA:IEEE Computer Society Press,2016:770-778.
[33]FILIP R,GIORGOS T,ONDREJ C.Fine-tuning CNN ImageRetrieval with No Human Annotation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,41(7):1655-1668.
[34]IOFFE S,SZEGEDY C.Batch Normalization:Accelerating Deep Network Training by Reducing Internal Covariate Shift[C]//Proceedings of International Conference on Machine Learning.Lille,France:ACM Press,2015:448-456.
[35]LUO H,JIANG W,GU Y Z,et al.A Strong Baseline and Batch Normalization Neck for Deep Person Re-Identification[J].IEEE Transactions on Multimedia,2020,22(10):2597-2609.
[36]WU A C,ZHENG W S,YU H X,et al.RGB-Infrared Cross-Modality Person Re-identification[C]//Proceedings of International Conference on Computer Vision.Venice,Italy:IEEE Computer Society Press,2017:5390-5399.
[37]NGUYEN D T,HONG H G,KIM K W,et al.Person Recognition System Based on a Combination of Body Images From visible light and thermal cameras[J].Sensors,2017,17(3):605.
[38]ZHU J Q,ZENG H Q,LIAO S C,et al.Deep Hybrid Similarity Learning for Person Re-Identification[J].IEEE Transactions on Circuits and Systems for Video Technology,2018,28(11):3183-3193.
[39]SHEN F,ZHU J Q,ZHU X B,et al.An Efficient Multi-Resolution Network for Vehicle Re-Identification[J].IEEE Internet of Things Journal,2022,9(11):9049-9059.
[40]LI H,TANG M,LIN J W,et al.Cross-Modality Person Reidentification Fran Network Based on Improved Hard Triplet Loss[J].Computer Science,2020,47(10):180-186.
[41]XIE Y,SHEN F,ZHU J Q,et al.Viewpoint Robust Knowledge Distillation for Accelerating Vehicle Re-Identification[J].EURASIP Journal on Advances in Signal Processing,2021,2021(1):1-13.
[42]ZHONG Z,ZHENG L,KANG G,et al.Random Erasing Data Augmentation[C]//Proceedings of the AAAI Conference on Artificial Intelligence.New York,NY,USA:AAAI Press,2017:13001-13008.
[43]KRIZHEVSKY A,SUTSKEVER I,HINTON G E.ImagenetClassification with Deep Convolutional Neural Networks[C]//Proceedings of Annual Conference on Neural Information Processing Systems.Nevada,Las Vegas,USA:MIT Press,2012:1097-1105.
[44]FU C Y,HU Y B,WU X,et al.CM-NAS:Cross-Modality Neural Architecture Search for Visible-Infrared Person Re-Identification[C]//Proceedings of International Conference on Compu-ter Vision.Virtual:IEEE Computer Society Press,2021:11803-11812.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发