计算机科学 ›› 2021, Vol. 48 ›› Issue (12): 269-277.doi: 10.11896/jsjkx.210400121

• 计算机图形学&多媒体 • 上一篇    下一篇

基于多粒度区域相关深度特征学习的行人重识别

董虎胜1,2, 钟珊3, 杨元峰1,2, 孙逊1,2, 龚声蓉3   

  1. 1 江苏省现代企业信息化应用支撑软件工程技术研发中心 江苏 苏州215104
    2 苏州市职业大学计算机工程学院 江苏 苏州215104
    3 常熟理工学院计算机科学与工程学院 江苏 常熟215500
  • 收稿日期:2021-04-13 修回日期:2021-07-25 出版日期:2021-12-15 发布日期:2021-11-26
  • 通讯作者: 董虎胜(hsdong2012@gmail.com)
  • 基金资助:
    国家自然科学基金(61702055,61972059,61773272);江苏省自然科学基金(BK20191474,BK20191475,BK20161268);苏州市职业大学校级科研项目(SVU2021YY03)

Person Re-identification by Region Correlated Deep Feature Learning with Multiple Granularities

DONG Hu-sheng1,2, ZHONG Shan3, YANG Yuan-feng1,2, SUN Xun1,2, GONG Sheng-rong3   

  1. 1 Jiangsu Province Support Software Engineering R & D Center for Modern Information Technology Application in Enterprise,Suzhou,
    Jiangsu 215104,China
    2 School of Computer Engineering,Suzhou Vocational University,Suzhou,Jiangsu 215104,China
    3 School of Computer Science and Engineering,Changshu Institute of Technology,Changshu,Jiangsu 215500,China
  • Received:2021-04-13 Revised:2021-07-25 Online:2021-12-15 Published:2021-11-26
  • About author:DONG Hu-sheng,born in 1981,Ph.D,lecturer,is a member of China Compu-ter Federation.His main research in-terests include computer vision,machine learning,and deep learning.
  • Supported by:
    National Natural Science Foundation of China(61702055,61972059,61773272),Natural Science Foundation of Jiangsu Province(BK20191474,BK20191475,BK20161268) and Research Funds of Suzhou Vocational University(SVU2021YY03).

摘要: 在对行人重识别的研究中,联合使用从图像中提取的全身与局部特征已经成为当前的主流方法。但是许多基于深度学习的重识别模型在提取局部特征时忽略了它们在空间上的相互联系,当不同行人具有局部相似的外观时,这些局部特征的辨别能力会受到很大影响。针对该问题,提出了一种学习多粒度区域相关特征的行人重识别方法。该方法在对骨干网络提取的卷积特征张量作不同粒度的区域划分后,设计了区域相关子网络模块来学习融入空间结构关系的各局部区域特征。在区域相关子网络模块中,为了赋予局部特征与其他区域相关联的空间结构信息,综合利用了平均池化运算的空间保持能力与最大池化运算的性能优势。通过对当前特征和其他各区域的局部特征进行联合处理,使各局部特征间产生很强的空间相关性,提升了特征判别能力。在区域相关子网络模块的设计上,采用了与深度残差网络相同的短路连接结构,使得网络更易于优化。最后,由全身特征与使用区域相关子网络增强后的各局部区域特征联合实现行人重识别。Market-1501,CUHK03,DukeMTMC-reID 3个公开数据集上的实验结果表明,所提算法取得了优于当前主流算法的行人身份匹配准确率,具有非常优秀的重识别性能。

关键词: 池化操作, 区域相关网络, 深度学习, 特征表达, 行人重识别

Abstract: Extracting both global and local features from pedestrian images has become the mainstream inperson re-identification.While among most of current deep learning based person re-identification models,the relations between adjacent body parts are seldom taken into consideration during extracting local features.This may decay the capability of distinguishing different persons when they share similar attributes of local regions.To address this problem,a novel method is proposed to learn region correlated deep features for person re-identification.In our model,the output feature map of backbone network is partitioned with multiple granularities first.And then the structure information preserved local features are learned via a new designed Region Correlated Network (RCNet) module.The RCNet makes full use of the structure maintenance of average pooling and the performance advantage of max pooling,endowing local features with rich structural information.By jointly processing current feature and local features from other regions,they are strongly related to each other due to the spatial correlation.As a result,the discrimination of them is significantly enhanced.For better optimization of the whole network,the shortcut connection in deep residual networks is also employed in the architecture of RCNet.Finally,the re-identification is conducted with both global features and the local features with structural information incorporated.Experimental results show that the proposed method achieves higher matching accuracies in comparison with existing approaches on the public Market-1501,CUHK03 and DukeMTMC-reID datasets,demonstrating favorable re-identification performance.

Key words: Deep Learning, Feature representation, Person re-identification, Pooling operation, Region correlated network

中图分类号: 

  • TP391
[1]KHAWAR I.Person Search:New Paradigm of Person Re-Identification:A Survey And Outlook of Recent Works [J].Image and Vision Computing,2020,101:1-11.
[2]SRIKRISHNAK,GOU M R,WU Z Y,et al.A Systematic Eva- luation And Benchmark for Person Re-Identification:Features,Metrics,And Datasets [J].IEEE Transactions on Pattern Ana-lysis and Machine Intelligence,2018,41(3):523-536.
[3]LECUN Y,BENGIO Y,HINTON G.Deep learning [J].Nature,2015,521(7553):436-444.
[4]LI W,ZHAO R,XIAO T,et al.DeepReID:Deep Filter Pairing Neural Network for Person Re-identification[C]//IEEE Confe-rence on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2014:152-159.
[5]ZHENG L,SHEN L Y,TIAN L,et al.Scalable Person Re-identification:A Benchmark[C]//IEEE International Conference On Computer Vision (ICCV).IEEE Computer Society,2015:1116-1124.
[6]SUN Y F,ZHENG L,DENG W J,et al.SVDNet for Pedestrian Retrieval[C]//IEEE International Conference on Computer Vision (ICCV).IEEE Computer Society,2017:3800-3808.
[7]LUO H,JIANG W,ZHANG X,et al.AlignedReID++:Dynamically Matching Local Information for Person Re-Identification [J].Pattern Recognition,2019,94:53-61.
[8]WANG G S,YUAN Y F,CHEN X,et al.Learning Discriminative Features with Multiple Granularities for Person Re-identification[C]//ACM International Conference on Multimedia.2018:274-282.
[9]SUN Y F,ZHENG L,YANG Y,et al.Beyond Part Models:Per- son Retrieval with Refined Part Pooling (and A Strong Convolutional Baseline)[C]//European Conference on Computer Vision (ECCV).2018:480-496.
[10]YE M,SHEN J B,LIN G J,et al.Deep Learning for Person Re-Identification:A Survey And Outlook[J].arXiv:2001.04193v1.
[11]XIAO T,LI H S,OUYANG W L,et al.Learning Deep Feature Representations with Domain Guided Dropout for Person Re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2016:1249-1258.
[12]ZHENG F,DENG C,SUN X,et al.Pyramidal Person Re-identification via Multi-loss Dynamic Training[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2019:8514-8522.
[13]ZHANG Z Z,LAN C L,ZENG W J,et al.Densely Semantically Aligned Person Re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2019:667-676.
[14]HE K M,ZHANG X Y,REN S Q,et al.Deep Residual Learning For Image Recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2016:770-778.
[15]ZHENG Z D,ZHENG L,YANG Y.Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in Vitro[C]//IEEE International Conference on Computer Vision (ICCV).IEEE Computer Society,2017:3754-3762.
[16]AHMED E,JONES M,MARKS T K.An Improved Deep Learning Architecture for Person Re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2015:3908-3916.
[17]HOU R B,MA B P,CHANG H,et al.Interaction-and-aggregation Network for Person Re-identification[C]//IEEE Confe-rence on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2019:9317-9326.
[18]CHENG D,GONG Y H,ZHOU S P,et al.Person Re-identification by Multi-channel Parts-based CNN with Improved Triplet Loss Function[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2016:1335-1344.
[19]SUN Y F,XU Q,LI Y L,et al.Perceive Where To Focus:Learning Visibility-aware Part-level Features for Partial Person Re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2019:393-402.
[20]WANG G S,YUAN Y F,LI J W,et al.Receptive Multi-granularity Representation for Person Re-Identification[J].IEEE Transaction on Image Processing,2020,29:6096-6109.
[21]SONG C F,HUANG Y,OUYANG W L,et al.Mask-guided Contrastive Attention Model for Person Re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2018:1179-1188.
[22]XU J,ZHAO R,ZHU F,et al.Attention-aware Compositional Network for Person Re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2018:2119-2128.
[23]ZHAO H Y,TIAN M Q,SUN S Y,et al.Spindle Net:Person Re-identification with Human Body Region Guided Feature Decomposition and Fusion[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2017:1077-1085.
[24]CHEN X S,FU C M,ZHAO Y.Salience-Guided Cascaded Suppression Network for Person Re-identification [C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2020:3300-3310.
[25]MIAO J X,WU Y,LIU P,et al.Pose-guided Feature Alignment for Occluded Person Re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2019:542-551.
[26]ZHANG Z Z,LAN C L,ZENG W J,et al.Relation-Aware Glo- bal Attention for Person Re-identification[C]//IEEE Confe-rence on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2020:3186-3195.
[27]FU Y,WEI Y C,ZHOU Y Q,et al.Horizontal Pyramid Matching for Person Re-identification[C]//AAAI Conference on Artificial Intelligence.2019,33:8295-8302.
[28]PARK H,HAM B.Relation Network for Person Re-identification[C]//AAAI Conference on Artificial Intelligence.2020:11839-11847.
[29]HERMANS A,BEYER L,LEIBE B.In defense of the triplet loss for person re-identification [J].arXiv:1703.07737.
[30]ZHONG Z,ZHENG L,CAO D L,et al.Re-ranking Person Re-identification with K-reciprocal Encoding[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2017:1318-1327.
[31]DENG J,DONG W,SOCHER R,et al.ImageNet:A Large-scale Hierarchical Image Database[C]//IEEE Conference on Compu-ter Vision and Pattern Recognition (CVPR).IEEE Computer Society,2009:248-255.
[32]ZHONG Z,ZHENG L,KANG G L,et al.Random Erasing Data Augmentation[C]//AAAI Conference on Artificial Intelligence.2020:13001-13008.
[33]LUO H,JIANG W,GU Y Z,et al.A Strong Baseline and Batch Normalization Neck for Deep Person Re-Identification [J].IEEE Transactions on Multimedia,2020,22(10):2597-2609.
[34]LI W,ZHU X T,GONG S G.Harmonious Attention Network for Person Re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2018:2285-2294.
[35]SARFRAZ M S,SCHUMANN A,EBERLE A,et al.A Pose-sensitive Embedding for Person Re-identification with Expanded Cross Neighborhood Re-ranking[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2018:420-429.
[36]ZHAO L M,LI X,ZHUANG Y T,et al.Deeply-learned Part- aligned Representations for Person Re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2017:3219-3228.
[1] 饶志双, 贾真, 张凡, 李天瑞.
基于Key-Value关联记忆网络的知识图谱问答方法
Key-Value Relational Memory Networks for Question Answering over Knowledge Graph
计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[2] 汤凌韬, 王迪, 张鲁飞, 刘盛云.
基于安全多方计算和差分隐私的联邦学习方案
Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy
计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[3] 徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺.
时序知识图谱表示学习
Temporal Knowledge Graph Representation Learning
计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[4] 王剑, 彭雨琦, 赵宇斐, 杨健.
基于深度学习的社交网络舆情信息抽取方法综述
Survey of Social Network Public Opinion Information Extraction Based on Deep Learning
计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[5] 郝志荣, 陈龙, 黄嘉成.
面向文本分类的类别区分式通用对抗攻击方法
Class Discriminative Universal Adversarial Attack for Text Classification
计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[6] 姜梦函, 李邵梅, 郑洪浩, 张建朋.
基于改进位置编码的谣言检测模型
Rumor Detection Model Based on Improved Position Embedding
计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[7] 孙奇, 吉根林, 张杰.
基于非局部注意力生成对抗网络的视频异常事件检测方法
Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection
计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[8] 胡艳羽, 赵龙, 董祥军.
一种用于癌症分类的两阶段深度特征选择提取算法
Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification
计算机科学, 2022, 49(7): 73-78. https://doi.org/10.11896/jsjkx.210500092
[9] 程成, 降爱莲.
基于多路径特征提取的实时语义分割方法
Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction
计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[10] 孟月波, 穆思蓉, 刘光辉, 徐胜军, 韩九强.
基于向量注意力机制GoogLeNet-GMP的行人重识别方法
Person Re-identification Method Based on GoogLeNet-GMP Based on Vector Attention Mechanism
计算机科学, 2022, 49(7): 142-147. https://doi.org/10.11896/jsjkx.210600198
[11] 侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木.
中文预训练模型研究进展
Advances in Chinese Pre-training Models
计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[12] 周慧, 施皓晨, 屠要峰, 黄圣君.
基于主动采样的深度鲁棒神经网络学习
Robust Deep Neural Network Learning Based on Active Sampling
计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
[13] 苏丹宁, 曹桂涛, 王燕楠, 王宏, 任赫.
小样本雷达辐射源识别的深度学习方法综述
Survey of Deep Learning for Radar Emitter Identification Based on Small Sample
计算机科学, 2022, 49(7): 226-235. https://doi.org/10.11896/jsjkx.210600138
[14] 祝文韬, 兰先超, 罗唤霖, 岳彬, 汪洋.
改进Faster R-CNN的光学遥感飞机目标检测
Remote Sensing Aircraft Target Detection Based on Improved Faster R-CNN
计算机科学, 2022, 49(6A): 378-383. https://doi.org/10.11896/jsjkx.210300121
[15] 王建明, 陈响育, 杨自忠, 史晨阳, 张宇航, 钱正坤.
不同数据增强方法对模型识别精度的影响
Influence of Different Data Augmentation Methods on Model Recognition Accuracy
计算机科学, 2022, 49(6A): 418-423. https://doi.org/10.11896/jsjkx.210700210
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!