Computer Science

Computer Science ›› 2023, Vol. 50 ›› Issue (6A): 221100046-8.doi: 10.11896/jsjkx.221100046

• Image Processing & Multimedia Technology • Previous Articles     Next Articles

Metric Regularized Infrared and Visible Cross-modal Person Re-identification

WU Hanxiao1, ZHAO Qianqian1, ZHU Jianqing2,4, ZENG Huanqiang2, DU Jixiang3, LIAO Yun4   

  1. 1 College of Information Science and Engineering,Huaqiao University,Xiamen,Fujian 361021,China;
    2 College of Engineering,Huaqiao University,Quanzhou,Fujian 362011,China;
    3 College of Computer Science and Technology,Huaqiao University,Xiamen,Fujian 361021,China;
    4 Xiamen Yealink Network Technology Co.,LTD,Xiamen,Fujian 361015,China
  • Online:2023-06-10 Published:2023-06-12
  • About author:WU Hanxiao,born in 1997,postgra-duate.Her main research interest is object re-identification. ZHU Jianqing,born in 1987,professor,master supervisor.His main research interests include deep learning,machine vision and pattern recognition.
  • Supported by:
    Natural Science Foundation for Outstanding Young Scholars of Fujian Province(2022J06023) and National Natural Science Foundation of China(61976098).

PDF (PC)

262

Abstract: Infrared and visible cross-modal person re-identification plays an important role in improving the all-day combat capability of intelligent video surveillance systems.Existing methods usually focus on the alignment of cross-modal features,and neglect the metric alignment among multiple modalities,resulting in re-identification lacking robustness to modal changes.For that,this paper proposes a metric regularized infrared and visible cross-modal person re-identification.First,this paper designs a metric regularized loss function to constrain the difference among matching behaviors under different modal retrieval modes to improve the robustness.Second,considering that the number of infrared images is less than that of visible images in actual surveillance scenes,this paper applies the modal data proportion to modify the cross-entropy function to reduce the adverse effect of the imba-lance between different modalities.Experimental results show the superiority of the proposed method,e.g.,using visible images to retrieval infrared images,the rank-1 identification rate reaches 89.52% on the RegDB dataset.

Key words: Metric regularized loss function, Metric alignment, Cross-modal, Person re-identification, Intelligent video surveillance systems

CLC Number: 

  • TP391.41
[1]WANG X D,ZHENG Z D,HE Y,et al.Soft Person Reidentification Network Pruning via Bloc-kwise Adjacent Filter Decaying[J].IEEE Transaction on Cybernetics,2022,52(12):13293-13307.
[2]XU F R,MA B P,CHANG H,et al.PRDP:Person Re-identfication with Dirty and Poor Data[J].IEEE Transactions on Cybernetics,2022,52(10):11014-11026..
[3]ZHANG X F,SONG B.A Person Reidentification Method Basedon Improved Triple Loss and Feature Fusion[J].Journal of Computer-Aided Computer Science,2021,48(9):146-152.
[4]GAO Z,GAO L S,ZHANG H,et al.DCR:a Unified Framework for Holistic/Partial Person Reid[J].IEEE Transactions on Multimedia,2020,23:3332-3345.
[5]HUANG Y,WU Q,XU J S,et al.Alleviating Modality BiasTraining for Infrared-Visible Person Re-Identification[J].IEEE Transactions on Multimedia,2022,24:1570-1582.
[6]YE M,WANG Z,LAN X Y,et al.Visible Thermal Person Re-Identification via Dual-Constrained Top-Ranking[C]//Procee-dings of the International Joint Conference on Artificial Intelligence.Stockholm,Sweden:Morgan Kaufmann,2018:1092-1099.
[7]YE M,LAN X Y,LENG Q M,et al.Cross-Modality Person Re-identification via Modality-Aware Collaborative Ensemble Learning[J].IEEE Transactions on Image Processing,2020,29:9387-9399.
[8]CHENG Y Z,XIAO G Q,TANG X Q,et al.Two-Phase Feature Fusion Network for Visible-Infrared Person Re-Identification[C]//Proceedings of International Conference on Image Processing.Anchorage,Alaska,USA:IEEE Computer Society Press,2021:1149-1153.
[9]LIU H J,TAN X H,ZHOU X C.Parameter Sharing Exploration and Hetero-Center Triplet Loss for Visible-Thermal Person Re-Identification[J].IEEE Transactions on Multimedia,2021,23:4414-4425.
[10]LIU H,MA S,XIA D,et al.SFANet:a Spectrum-Aware Feature Augmentation Network for Visible-Infrared Person Reidentification[J].IEEE Transactions on Neural Networks and Learning Systems,2023,34(4):1958-1971.
[11]YE M,SHEN J B,LIN G J,et al.Deep Learning for Person Re-Identification:A Survey and Outlook[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2022,44(6):2872-2893.
[12]YE M,SHEN J B,CRANDALL D J,et al.Dynamic Dual-Attentive Aggregation Learning for Visible-Infrared Person Re-Identification[C]//Proceedings of European Conference on Compu-ter Vision.Vitrual:Springer,2020:229-247.
[13]YE M,LAN X Y,WANG Z,et al.Bi-Directional Center-Con-strained Top-Ranking for Visible-Thermal Person Re-Identification[J].IEEE Transactions on Information Forensics and Security,2020,15:407-419.
[14]ZHANG C,LIU H,GUO W,et al.Multi-Scale Cascading Network with Compact Feature Learning for RGB-infrared Person Re-Identification[C]//Proceedings of International Conference on Pattern Recognition.Milan,Italy:IEEE Computer Society Press,2020:8679-8686.
[15]HAO Y,WANG N N,LI J,et al.HSME:Hypersphere ManifoldEmbedding for Visible Thermal Person Re-identification[C]//Proceedings of AAAI Conference on Artificial Intelligence.Honolulu,Hawaii,USA:AAAI Press,2019:8385-8392.
[16]WEI Z Y,YANG X,WANG N N,et al.Flexible Body Partition-Based Adversarial Learning for Visible-Infrared Person Re-Identification[J].IEEE Transactions on Neural Networks and Learning Systems,2022,33(9):4676-4687.
[17]FENG Z,LAI J,XIE X.Learning Modality-Specific Representations for Visible-Infrared Person Re-Identification[J].IEEE Transactions on Image Processing,2019,29:579-590.
[18]PARK H,LEE S H,LEE J,et al.Learning by Aligning:Visible-Infrared Person Re-identification Using Cross-Modal Correspondences[C]//Proceedings of International Conference on Computer Vision.Virtual:IEEE Computer Society Press,2021:12046-12055.
[19]ZHANG L Y,DU G D,LIU F,et al.Global-Local MultipleGranularity Learning for Cross-Modality Visible-Infrared Person Reidentification[J/OL].IEEE Transactions on Neural Networks and Learning Systems(Early Access),2021.https://ieeexplore.ieee.org/document/9457243.
[20]ZHONG X,LU T Y,HUANG W X,et al.Grayscale Enhancement Colorization Network for Visible-Infrared Person Re-Identification[J].IEEE Transactions on Circuits and Systems for Video Technology,2022,32(3):1418-1430.
[21]DAI H P,XIE Q,MA Y C,et al.RGB-Infrared Person Re-identification via Image Modality Conversion[C]//Proceedings of International Conference on Pattern Recognition.Milan,Italy:IEEE Computer Society Press,2020:592-598.
[22]DAI P Y,JI R R,WANG H B,et al.Cross-Modality Person Re-identification with Generative Adversarial Training[C]//Proceedings of International Joint Conference on Artificial Intelligence.Stockholm,Sweden:Morgan Kaufmann,2018:677-683.
[23]WANG G A,ZHANG T Z,YANG Y,et al.Cross-ModalityPaired-Images Generation for RGB-Infrared Person Re-Identification[C]//Proceedings of AAAI Conference on Artificial Intelligence.New York,NY,USA:AAAI Press,2020.12144-12151.
[24]WANG G A,ZHANG T Z,CHENG J,et al.RGB-infraredCross-Modality Person Re-identification Via Joint Pixel and Feature Alignment[C]//Proceedings of International Conference on Computer Vision.Seoul,Korea:IEEE Computer Society Press,2019:3622-3631.
[25]LIU H,MIAO Z L,YANG B,et al.A Base-Derivative Framework for Cross-Modality RGB-infrared Person Re-Identification[C]//Proceedings of International Conference on Pattern Re-cognition.Milan,Italy:IEEE Computer Society Press,2020:7640-7646.
[26]HU B Y,LIU J W,ZHA Z J.Adversarial Disentanglement and Correlation Network for RGB-infrared Person Re-Identification[C]//Proceedings of International Conference on Multimedia and Expo.Shenzhen,China:IEEE Computer Society Press,2021:1-6.
[27]WEI Z Y,YANG X,WANG N N,et al.Syncretic Modality Collaborative Learning for Visible Infrared Person Re-Identification[C]//Proceedings of International Conference on Computer Vision.Virtual:IEEE Computer Society Press,2021:225-243.
[28]WU D,WANG C,WU Y,et al.Attention Deep Model withMulti-Scale Deep Supervision for Person Re-Identification[J].IEEE Transactions on Emerging Topics in Computational Intelligence,2021,5(1):70-78.
[29]ZHONG Z,ZHENG L,ZHENG Z D,et al.CamStyle:A Novel Data Augmentation Method for Person Re-Identification[J].IEEE Transactions on Image Processing,2019.28(3):1176-1190.
[30]SUN Y,WANG X,TANG X.Deep Learning Face Representation from Predicting 10,000 Classes[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Columbus,OH,USA:IEEE Computer Society Press,2014:1891-1898.
[31]HAO X,ZHAO S Y,YE M,et al.Cross-Modality Person Re-Identification via Modality Confusion and Center Aggregation[C]//Proceedings of International Conference on Computer Vision.Virtual:IEEE Computer Society Press,2021:16383-16392.
[32]HE K M,ZHANG X Y,REN S Q,et al.Deep Residual Learning for Image Recognition[C]//Proceedings of Conference on Computer Vision and Pattern Recognition.Las Vegas,NV,USA:IEEE Computer Society Press,2016:770-778.
[33]FILIP R,GIORGOS T,ONDREJ C.Fine-tuning CNN ImageRetrieval with No Human Annotation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,41(7):1655-1668.
[34]IOFFE S,SZEGEDY C.Batch Normalization:Accelerating Deep Network Training by Reducing Internal Covariate Shift[C]//Proceedings of International Conference on Machine Learning.Lille,France:ACM Press,2015:448-456.
[35]LUO H,JIANG W,GU Y Z,et al.A Strong Baseline and Batch Normalization Neck for Deep Person Re-Identification[J].IEEE Transactions on Multimedia,2020,22(10):2597-2609.
[36]WU A C,ZHENG W S,YU H X,et al.RGB-Infrared Cross-Modality Person Re-identification[C]//Proceedings of International Conference on Computer Vision.Venice,Italy:IEEE Computer Society Press,2017:5390-5399.
[37]NGUYEN D T,HONG H G,KIM K W,et al.Person Recognition System Based on a Combination of Body Images From visible light and thermal cameras[J].Sensors,2017,17(3):605.
[38]ZHU J Q,ZENG H Q,LIAO S C,et al.Deep Hybrid Similarity Learning for Person Re-Identification[J].IEEE Transactions on Circuits and Systems for Video Technology,2018,28(11):3183-3193.
[39]SHEN F,ZHU J Q,ZHU X B,et al.An Efficient Multi-Resolution Network for Vehicle Re-Identification[J].IEEE Internet of Things Journal,2022,9(11):9049-9059.
[40]LI H,TANG M,LIN J W,et al.Cross-Modality Person Reidentification Fran Network Based on Improved Hard Triplet Loss[J].Computer Science,2020,47(10):180-186.
[41]XIE Y,SHEN F,ZHU J Q,et al.Viewpoint Robust Knowledge Distillation for Accelerating Vehicle Re-Identification[J].EURASIP Journal on Advances in Signal Processing,2021,2021(1):1-13.
[42]ZHONG Z,ZHENG L,KANG G,et al.Random Erasing Data Augmentation[C]//Proceedings of the AAAI Conference on Artificial Intelligence.New York,NY,USA:AAAI Press,2017:13001-13008.
[43]KRIZHEVSKY A,SUTSKEVER I,HINTON G E.ImagenetClassification with Deep Convolutional Neural Networks[C]//Proceedings of Annual Conference on Neural Information Processing Systems.Nevada,Las Vegas,USA:MIT Press,2012:1097-1105.
[44]FU C Y,HU Y B,WU X,et al.CM-NAS:Cross-Modality Neural Architecture Search for Visible-Infrared Person Re-Identification[C]//Proceedings of International Conference on Compu-ter Vision.Virtual:IEEE Computer Society Press,2021:11803-11812.
[1] DAI Xuesong, LI Xiaohong, ZHANG Jingjing, QI Meibin, LIU Yimin. Unsupervised Domain Adaptive Pedestrian Re-identification Based on Counterfactual AttentionLearning [J]. Computer Science, 2023, 50(7): 160-166.
[2] ZHANG Shuaiyu, PENG Li, DAI Feifei. Person Re-identification Method Based on Progressive Attention Pyramid [J]. Computer Science, 2023, 50(6A): 220200084-8.
[3] ZHANG Changfan, MA Yuanyuan, LIU Jianhua, HE Jing. Dual Gating-Residual Feature Fusion for Image-Text Cross-modal Retrieval [J]. Computer Science, 2023, 50(6A): 220700030-7.
[4] QIN Liang, XIE Liang, CHEN Shengshuang, XU Haijiao. Online Semi-supervised Cross-modal Hashing Based on Anchor Graph Classification [J]. Computer Science, 2023, 50(6): 183-193.
[5] CHENG Haiyang, ZHANG Jianxin, SUN Qisen, ZHANG Qiang, WEI Xiaopeng. Deep Cross-modal Information Fusion Network for Stock Trend Prediction [J]. Computer Science, 2023, 50(5): 128-136.
[6] LI Xiang, FAN Zhiguang, LI Xuexiang, ZHANG Weixing, YANG Cong, CAO Yangjie. Survey of Visual Question Answering Based on Deep Learning [J]. Computer Science, 2023, 50(5): 177-188.
[7] HE Chao, CHEN Jinjie, JIN Zhao, LEI Yinjie. Automatic Modulation Recognition Method Based on Multimodal Time-Frequency Feature Fusion [J]. Computer Science, 2023, 50(4): 226-232.
[8] YANG Xiaoyu, LI Chao, CHEN Shunyao, LI Haoliang, YIN Guangqiang. Text-Image Cross-modal Retrieval Based on Transformer [J]. Computer Science, 2023, 50(4): 141-148.
[9] LIU Yang, LIU Ruijia, ZHOU Liming, ZUO Xianyu, YANG Wei, ZHOU Yi. Thoughts on Development and Research of Science,Technology and Engineering Application of Brain & Mind-inspired Computing [J]. Computer Science, 2023, 50(2): 364-373.
[10] NIE Xiu-shan, PAN Jia-nan, TAN Zhi-fang, LIU Xin-fang, GUO Jie, YIN Yi-long. Overview of Natural Language Video Localization [J]. Computer Science, 2022, 49(9): 111-122.
[11] CHEN Kun-feng, PAN Zhi-song, WANG Jia-bao, SHI Lei, ZHANG Jin. Moderate Clothes-Changing Person Re-identification Based on Bionics of Binocular Summation [J]. Computer Science, 2022, 49(8): 165-171.
[12] ZENG Zhi-xian, CAO Jian-jun, WENG Nian-feng, JIANG Guo-quan, XU Bin. Fine-grained Semantic Association Video-Text Cross-modal Entity Resolution Based on Attention Mechanism [J]. Computer Science, 2022, 49(7): 106-112.
[13] MENG Yue-bo, MU Si-rong, LIU Guang-hui, XU Sheng-jun, HAN Jiu-qiang. Person Re-identification Method Based on GoogLeNet-GMP Based on Vector Attention Mechanism [J]. Computer Science, 2022, 49(7): 142-147.
[14] YANG Xiao-yu, YIN Kang-ning, HOU Shao-qi, DU Wen-yi, YIN Guang-qiang. Person Re-identification Based on Feature Location and Fusion [J]. Computer Science, 2022, 49(3): 170-178.
[15] ZHANG Bin, LIU Chang-hong, ZENG Sheng, JIE An-quan. Speech-driven Personal Style Gesture Generation Method Based on Spatio-Temporal GraphConvolutional Networks [J]. Computer Science, 2022, 49(11A): 210900094-5.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.