计算机科学 ›› 2021, Vol. 48 ›› Issue (12): 269-277.doi: 10.11896/jsjkx.210400121

• 计算机图形学&多媒体 • 上一篇    下一篇

基于多粒度区域相关深度特征学习的行人重识别

董虎胜1,2, 钟珊3, 杨元峰1,2, 孙逊1,2, 龚声蓉3   

  1. 1 江苏省现代企业信息化应用支撑软件工程技术研发中心 江苏 苏州215104
    2 苏州市职业大学计算机工程学院 江苏 苏州215104
    3 常熟理工学院计算机科学与工程学院 江苏 常熟215500
  • 收稿日期:2021-04-13 修回日期:2021-07-25 出版日期:2021-12-15 发布日期:2021-11-26
  • 通讯作者: 董虎胜(hsdong2012@gmail.com)
  • 基金资助:
    国家自然科学基金(61702055,61972059,61773272);江苏省自然科学基金(BK20191474,BK20191475,BK20161268);苏州市职业大学校级科研项目(SVU2021YY03)

Person Re-identification by Region Correlated Deep Feature Learning with Multiple Granularities

DONG Hu-sheng1,2, ZHONG Shan3, YANG Yuan-feng1,2, SUN Xun1,2, GONG Sheng-rong3   

  1. 1 Jiangsu Province Support Software Engineering R & D Center for Modern Information Technology Application in Enterprise,Suzhou,
    Jiangsu 215104,China
    2 School of Computer Engineering,Suzhou Vocational University,Suzhou,Jiangsu 215104,China
    3 School of Computer Science and Engineering,Changshu Institute of Technology,Changshu,Jiangsu 215500,China
  • Received:2021-04-13 Revised:2021-07-25 Online:2021-12-15 Published:2021-11-26
  • About author:DONG Hu-sheng,born in 1981,Ph.D,lecturer,is a member of China Compu-ter Federation.His main research in-terests include computer vision,machine learning,and deep learning.
  • Supported by:
    National Natural Science Foundation of China(61702055,61972059,61773272),Natural Science Foundation of Jiangsu Province(BK20191474,BK20191475,BK20161268) and Research Funds of Suzhou Vocational University(SVU2021YY03).

摘要: 在对行人重识别的研究中,联合使用从图像中提取的全身与局部特征已经成为当前的主流方法。但是许多基于深度学习的重识别模型在提取局部特征时忽略了它们在空间上的相互联系,当不同行人具有局部相似的外观时,这些局部特征的辨别能力会受到很大影响。针对该问题,提出了一种学习多粒度区域相关特征的行人重识别方法。该方法在对骨干网络提取的卷积特征张量作不同粒度的区域划分后,设计了区域相关子网络模块来学习融入空间结构关系的各局部区域特征。在区域相关子网络模块中,为了赋予局部特征与其他区域相关联的空间结构信息,综合利用了平均池化运算的空间保持能力与最大池化运算的性能优势。通过对当前特征和其他各区域的局部特征进行联合处理,使各局部特征间产生很强的空间相关性,提升了特征判别能力。在区域相关子网络模块的设计上,采用了与深度残差网络相同的短路连接结构,使得网络更易于优化。最后,由全身特征与使用区域相关子网络增强后的各局部区域特征联合实现行人重识别。Market-1501,CUHK03,DukeMTMC-reID 3个公开数据集上的实验结果表明,所提算法取得了优于当前主流算法的行人身份匹配准确率,具有非常优秀的重识别性能。

关键词: 行人重识别, 深度学习, 特征表达, 池化操作, 区域相关网络

Abstract: Extracting both global and local features from pedestrian images has become the mainstream inperson re-identification.While among most of current deep learning based person re-identification models,the relations between adjacent body parts are seldom taken into consideration during extracting local features.This may decay the capability of distinguishing different persons when they share similar attributes of local regions.To address this problem,a novel method is proposed to learn region correlated deep features for person re-identification.In our model,the output feature map of backbone network is partitioned with multiple granularities first.And then the structure information preserved local features are learned via a new designed Region Correlated Network (RCNet) module.The RCNet makes full use of the structure maintenance of average pooling and the performance advantage of max pooling,endowing local features with rich structural information.By jointly processing current feature and local features from other regions,they are strongly related to each other due to the spatial correlation.As a result,the discrimination of them is significantly enhanced.For better optimization of the whole network,the shortcut connection in deep residual networks is also employed in the architecture of RCNet.Finally,the re-identification is conducted with both global features and the local features with structural information incorporated.Experimental results show that the proposed method achieves higher matching accuracies in comparison with existing approaches on the public Market-1501,CUHK03 and DukeMTMC-reID datasets,demonstrating favorable re-identification performance.

Key words: Person re-identification, Deep Learning, Feature representation, Pooling operation, Region correlated network

中图分类号: 

  • TP391
[1]KHAWAR I.Person Search:New Paradigm of Person Re-Identification:A Survey And Outlook of Recent Works [J].Image and Vision Computing,2020,101:1-11.
[2]SRIKRISHNAK,GOU M R,WU Z Y,et al.A Systematic Eva- luation And Benchmark for Person Re-Identification:Features,Metrics,And Datasets [J].IEEE Transactions on Pattern Ana-lysis and Machine Intelligence,2018,41(3):523-536.
[3]LECUN Y,BENGIO Y,HINTON G.Deep learning [J].Nature,2015,521(7553):436-444.
[4]LI W,ZHAO R,XIAO T,et al.DeepReID:Deep Filter Pairing Neural Network for Person Re-identification[C]//IEEE Confe-rence on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2014:152-159.
[5]ZHENG L,SHEN L Y,TIAN L,et al.Scalable Person Re-identification:A Benchmark[C]//IEEE International Conference On Computer Vision (ICCV).IEEE Computer Society,2015:1116-1124.
[6]SUN Y F,ZHENG L,DENG W J,et al.SVDNet for Pedestrian Retrieval[C]//IEEE International Conference on Computer Vision (ICCV).IEEE Computer Society,2017:3800-3808.
[7]LUO H,JIANG W,ZHANG X,et al.AlignedReID++:Dynamically Matching Local Information for Person Re-Identification [J].Pattern Recognition,2019,94:53-61.
[8]WANG G S,YUAN Y F,CHEN X,et al.Learning Discriminative Features with Multiple Granularities for Person Re-identification[C]//ACM International Conference on Multimedia.2018:274-282.
[9]SUN Y F,ZHENG L,YANG Y,et al.Beyond Part Models:Per- son Retrieval with Refined Part Pooling (and A Strong Convolutional Baseline)[C]//European Conference on Computer Vision (ECCV).2018:480-496.
[10]YE M,SHEN J B,LIN G J,et al.Deep Learning for Person Re-Identification:A Survey And Outlook[J].arXiv:2001.04193v1.
[11]XIAO T,LI H S,OUYANG W L,et al.Learning Deep Feature Representations with Domain Guided Dropout for Person Re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2016:1249-1258.
[12]ZHENG F,DENG C,SUN X,et al.Pyramidal Person Re-identification via Multi-loss Dynamic Training[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2019:8514-8522.
[13]ZHANG Z Z,LAN C L,ZENG W J,et al.Densely Semantically Aligned Person Re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2019:667-676.
[14]HE K M,ZHANG X Y,REN S Q,et al.Deep Residual Learning For Image Recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2016:770-778.
[15]ZHENG Z D,ZHENG L,YANG Y.Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in Vitro[C]//IEEE International Conference on Computer Vision (ICCV).IEEE Computer Society,2017:3754-3762.
[16]AHMED E,JONES M,MARKS T K.An Improved Deep Learning Architecture for Person Re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2015:3908-3916.
[17]HOU R B,MA B P,CHANG H,et al.Interaction-and-aggregation Network for Person Re-identification[C]//IEEE Confe-rence on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2019:9317-9326.
[18]CHENG D,GONG Y H,ZHOU S P,et al.Person Re-identification by Multi-channel Parts-based CNN with Improved Triplet Loss Function[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2016:1335-1344.
[19]SUN Y F,XU Q,LI Y L,et al.Perceive Where To Focus:Learning Visibility-aware Part-level Features for Partial Person Re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2019:393-402.
[20]WANG G S,YUAN Y F,LI J W,et al.Receptive Multi-granularity Representation for Person Re-Identification[J].IEEE Transaction on Image Processing,2020,29:6096-6109.
[21]SONG C F,HUANG Y,OUYANG W L,et al.Mask-guided Contrastive Attention Model for Person Re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2018:1179-1188.
[22]XU J,ZHAO R,ZHU F,et al.Attention-aware Compositional Network for Person Re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2018:2119-2128.
[23]ZHAO H Y,TIAN M Q,SUN S Y,et al.Spindle Net:Person Re-identification with Human Body Region Guided Feature Decomposition and Fusion[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2017:1077-1085.
[24]CHEN X S,FU C M,ZHAO Y.Salience-Guided Cascaded Suppression Network for Person Re-identification [C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2020:3300-3310.
[25]MIAO J X,WU Y,LIU P,et al.Pose-guided Feature Alignment for Occluded Person Re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2019:542-551.
[26]ZHANG Z Z,LAN C L,ZENG W J,et al.Relation-Aware Glo- bal Attention for Person Re-identification[C]//IEEE Confe-rence on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2020:3186-3195.
[27]FU Y,WEI Y C,ZHOU Y Q,et al.Horizontal Pyramid Matching for Person Re-identification[C]//AAAI Conference on Artificial Intelligence.2019,33:8295-8302.
[28]PARK H,HAM B.Relation Network for Person Re-identification[C]//AAAI Conference on Artificial Intelligence.2020:11839-11847.
[29]HERMANS A,BEYER L,LEIBE B.In defense of the triplet loss for person re-identification [J].arXiv:1703.07737.
[30]ZHONG Z,ZHENG L,CAO D L,et al.Re-ranking Person Re-identification with K-reciprocal Encoding[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2017:1318-1327.
[31]DENG J,DONG W,SOCHER R,et al.ImageNet:A Large-scale Hierarchical Image Database[C]//IEEE Conference on Compu-ter Vision and Pattern Recognition (CVPR).IEEE Computer Society,2009:248-255.
[32]ZHONG Z,ZHENG L,KANG G L,et al.Random Erasing Data Augmentation[C]//AAAI Conference on Artificial Intelligence.2020:13001-13008.
[33]LUO H,JIANG W,GU Y Z,et al.A Strong Baseline and Batch Normalization Neck for Deep Person Re-Identification [J].IEEE Transactions on Multimedia,2020,22(10):2597-2609.
[34]LI W,ZHU X T,GONG S G.Harmonious Attention Network for Person Re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2018:2285-2294.
[35]SARFRAZ M S,SCHUMANN A,EBERLE A,et al.A Pose-sensitive Embedding for Person Re-identification with Expanded Cross Neighborhood Re-ranking[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2018:420-429.
[36]ZHAO L M,LI X,ZHUANG Y T,et al.Deeply-learned Part- aligned Representations for Person Re-identification[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE Computer Society,2017:3219-3228.
[1] 董晓梅, 王蕊, 邹欣开. 面向推荐应用的差分隐私方案综述[J]. 计算机科学, 2021, 48(9): 21-35.
[2] 周新民, 胡宜桂, 刘文洁, 孙荣俊. 基于多模态多层级数据融合方法的城市功能识别研究[J]. 计算机科学, 2021, 48(9): 50-58.
[3] 钱梦薇, 过弋. 融合偏置深度学习的距离分解Top-N推荐算法[J]. 计算机科学, 2021, 48(9): 103-109.
[4] 徐涛, 田崇阳, 刘才华. 基于深度学习的人群异常行为检测综述[J]. 计算机科学, 2021, 48(9): 125-134.
[5] 张新峰, 宋博. 一种基于改进三元组损失和特征融合的行人重识别方法[J]. 计算机科学, 2021, 48(9): 146-152.
[6] 林椹尠, 张梦凯, 吴成茂, 郑兴宁. 利用生成对抗网络的人脸图像分步补全法[J]. 计算机科学, 2021, 48(9): 174-180.
[7] 黄晓生, 徐静. 基于PCANet的非下采样剪切波域多聚焦图像融合[J]. 计算机科学, 2021, 48(9): 181-186.
[8] 田野, 陈宏巍, 王法胜, 陈兴文. 室内移动机器人的SLAM算法综述[J]. 计算机科学, 2021, 48(9): 223-234.
[9] 谢良旭, 李峰, 谢建平, 许晓军. 基于融合神经网络模型的药物分子性质预测[J]. 计算机科学, 2021, 48(9): 251-256.
[10] 冯霞, 胡志毅, 刘才华. 跨模态检索研究进展综述[J]. 计算机科学, 2021, 48(8): 13-23.
[11] 王立梅, 朱旭光, 汪德嘉, 张勇, 邢春晓. 基于深度学习的民事案件判决结果分类方法研究[J]. 计算机科学, 2021, 48(8): 80-85.
[12] 郭琳, 李晨, 陈晨, 赵睿, 范仕霖, 徐星雨. 基于通道注意递归残差网络的图像超分辨率重建[J]. 计算机科学, 2021, 48(8): 139-144.
[13] 刘帅, 芮挺, 胡育成, 杨成松, 王东. 基于深度学习SuperGlue算法的单目视觉里程计[J]. 计算机科学, 2021, 48(8): 157-161.
[14] 王施云, 杨帆. 基于U-Net特征融合优化策略的遥感影像语义分割方法[J]. 计算机科学, 2021, 48(8): 162-168.
[15] 田嵩旺, 蔺素珍, 杨博. 基于多判别器的多波段图像自监督融合方法[J]. 计算机科学, 2021, 48(8): 185-190.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 赵彦明,季圣杰. 基于图像自身复杂视觉信息的特征提取算法与应用[J]. 计算机科学, 2014, 41(Z6): 234 -238 .
[2] 耿卫建,徐小龙,李玲娟,陈建新,杨庚. 智能小区用电数据模型研究[J]. 计算机科学, 2011, 38(Z10): 412 -415 .
[3] 曹义亲, 曹婷, 黄晓生. 基于àtrous-NSCT变换和区域特性的图像融合方法[J]. 计算机科学, 2019, 46(6): 270 -276 .
[4] 潘孝勤, 芦天亮, 杜彦辉, 仝鑫. 基于深度学习的语音合成与转换技术综述[J]. 计算机科学, 2021, 48(8): 200 -208 .
[5] 王俊, 王修来, 庞威, 赵鸿飞. 面向科技前瞻预测的大数据治理研究[J]. 计算机科学, 2021, 48(9): 36 -42 .
[6] 余力, 杜启翰, 岳博妍, 向君瑶, 徐冠宇, 冷友方. 基于强化学习的推荐研究综述[J]. 计算机科学, 2021, 48(10): 1 -18 .
[7] 王梓强, 胡晓光, 李晓筱, 杜卓群. 移动机器人全局路径规划算法综述[J]. 计算机科学, 2021, 48(10): 19 -29 .
[8] 高洪皓, 郑子彬, 殷昱煜, 丁勇. 区块链技术专题序言[J]. 计算机科学, 2021, 48(11): 1 -3 .
[9] 毛瀚宇, 聂铁铮, 申德荣, 于戈, 徐石成, 何光宇. 区块链即服务平台关键技术及发展综述[J]. 计算机科学, 2021, 48(11): 4 -11 .
[10] 徐堃, 付印金, 陈卫卫, 张亚男. 基于区块链的云存储安全研究进展[J]. 计算机科学, 2021, 48(11): 102 -115 .