有监督相似性保持的深度二阶哈希方法

doi:10.11896/jsjkx.210900021

摘要/Abstract

摘要： 近年来深度哈希方法因其存储效率高和查询速度快的优势在大规模图像检索领域受到了广泛关注。为改善深度成对有监督哈希方法在图像检索上的性能,从提高深度哈希获取图像特征的全局性和同类样本相似性角度出发,提出了一种有监督相似性保持的深度二阶哈希方法。该方法采用成对样本图像进行特征建模,并利用协方差估计来捕获样本图像的深度二阶信息,以获取具有良好全局表达能力的深度二阶哈希码;在此基础上,借鉴类哈希近似二值化来解决哈希映射过程中的非凸性问题,以更好地避免量化误差,同时基于多损失函数集成思想构建类别监督和相似性保持的联合约束,进而采用交替迭代的优化方式实现网络的端到端训练,最终确定样本图像的最优哈希码。在3个通用数据集上进行了广泛的实验,结果有效表明了所提出有监督相似性保持的深度二阶哈希方法的有效性。

关键词: 深度哈希, 二阶统计建模, 类别监督, 相似性保持, 图像检索

Abstract: Recently,deep hashing technology,with its advantages of high storage efficiency and quick query speed,has been widely investigated in the field of large-scale visual image retrieval.However,the fundamental image features obtained by current deep hashing methods mainly depend on the first-order statistics of deep convolutional features,and they seldom take the global structure into consideration,leading to the limitation of retrieval accuracy to a certain degree.Focusing on the global representation capability and the similarity of intra-class samples,this paper proposes a novel supervised similarity preserving deep second-order hashing (S2PDSoH) based on deep pairwise supervised hashing,gaining effective performance improvement in the image retrieval task.Based on the pair-wise deep hashing model,S2PDSoH first employs covariance estimation based on matrix power normalization method to capture the deep second-order information of sample images,so that hash codes can possess good global presentation ability.Then,to gain more robust hash codes,it further constructs a joint constraint of category supervision and similarity preservation motivated by the idea of multi-loss integration,followed by an alternate iteration optimization algorithm to realize the end-to-end training.Therefore,with the semantic information added to the dual channel deep second-order hashing framework,S2PDSoH establishes a mechanism for common constraints on category supervision and similarity maintenance.In addition,it also introduces a hash-like function to achieve the approximate binarization result of hash codes,which solves the problem of non-convexity and avoids the quantization error in the hash mapping process.Extensive experimental results on three commonly used data sets show the effectiveness of the proposed deep-order hashing method with supervised similarity preservation.

Key words: Deep hashing, Second-order statistic modeling, Category supervision, Similarity preserving, Image retrieval

中图分类号:

TP391.41

张建新, 吴悦, 张强, 魏小鹏. 有监督相似性保持的深度二阶哈希方法[J]. 计算机科学, 2022, 49(11A): 210900021-8. https://doi.org/10.11896/jsjkx.210900021

ZHANG Jian-xin, WU Yue, ZHANG Qiang, WEI Xiao-peng. Supervised Similarity Preserving Deep Second-order Hashing[J]. Computer Science, 2022, 49(11A): 210900021-8. https://doi.org/10.11896/jsjkx.210900021

参考文献

[1]RAWAT W,WANG Z H.Deep convolutional neural networks for image classification:acomprehensive review [J].Neural Computation,2017,29(9):2352-2449.
[2]WANG J D,ZHANG T,SONG J K,et al.A survey on learning to Hash [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,40(4):769-790.
[3]LIU Y,CHENG M,WANG F P,et al.Deep hashing image retrieval methods [J].Journal of Image and Graphics,2020,25(7):1296-1317.
[4]KE S C,ZHAO Y W,LI B C,et al.Image retrieval based on convolutional neural network and kernel-based supervised hashing [J].Acta Electronica Sinica,2017,45(1):157-163.
[5]XIA R K,PAN Y,LAI H J,et al.Supervised hashing for image retrieval via image representation learning[C]//Proceedings of the 28th AAAI Conference on Artificial Intelligence.Palo Alto:AAAI Press,2014:2156-2162.
[6]LIN K,YANG H F,HSIAO J,et al.Deep learning of binary hash codes for fast image retrieval[C]//Proceedings of the 28th IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2015:27-35.
[7]LAI H J,PAN Y,LIU Y,et al.Simultaneous feature learningand hash coding with deep neural networks[C]//Proceedings of the 28th IEEE Conference on Computer Vision and Pattern Re-cognition.Los Alamitos:IEEE Computer Society Press,2015:3270-3278.
[8]ZHAO F,HUANG Y Z,WANG L,et al.Deep semantic rankingbased hashing for multi-label image retrieval[C]//Proceedings of the 28th IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2015:1556-1564.
[9]FAN L X,NG K W,JU C,et al.Deep polarized network for supervised learning of accurate binary hashing codes[C]//Proceedings of the 2020 International Joint Conference on Artificial Intelligence.Japan:IJCAI,2020,825-831.
[10]WAN F,QIANG H P,LEI G B.Self-supervised deep discretehashing for image retrieval [J].Journal of Image and Graphics,2021,26(11):2659-2669.
[11]JIANG Q Y,LI W J.Discrete latent factor model for cross-modal hashing[J].IEEE Transactions on Image Processing,2019,28(7):3490-3501.
[12]IONESCU C,VANTZOS O,SMINCHISESCU C.Matrix Backpropagation for deep networks with structured layers[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2015:2965-2973.
[13]LIN T Y,ROYCHOWDHURY A,MAJI S.Bilinear convolu-tional neural networks for fine-grained visual recognition [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,40(6):1309-1322.
[14]LI P H,XIE J T,WANG Q L,et al.Is second-order information helpful for large-scale visual recognition? [C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2017:2089-2097.
[15]WANG Q L,XIE J T,ZUO W M,et al.Deep CNNs meet global covariance pooling:better representation and generalization[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2021,43(8):2582-2597.
[16]WU Y,SUN Q L,ZHANG J X,et al.Deep covariance estimation hashing for image retrieval[C]//Proceedings of the 26th IEEE International Conference on Image Processing.Los Alamitos:IEEE Computer Society Press,2019:2234-2238.
[17]WU Y,SUN Q L,HOU Y Q,et al.Deep covariance estimation hashing[J].IEEE Access,2019,7:113223-113234.
[18]LI W J,WANG S,KANG W C.Feature learning based deep supervised hashing with pairwise labels[C]///Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence.Palo Alto:AAAI Press,2016:1711-1717.
[19]LI Q,SUN Z N,HE R,et al.Deep supervised discrete hashing[C]//Proceedings of the 31st Annual Conference on Neural Information Processing Systems.Cambridge:MIT Press,2017:2482-2491.
[20]WANG X F,SHI Y,KITANI K M,et al.Deep supervised hashing with triplet labels[C]//Computer Vision－ACCV 2016.Lecture Notes in Computer Science.Cham:Springer,2017,10111:70-84.
[21]CHATFIELD K,SIMONYAN K,VEDALDI A,et al.Return of the devil in the details:Delving deep intoconvolutional nets[C]//Proceedings of the British Machine Vision Conference.Durham:BMVA Press,2014:1-11.
[22]HE K M,ZHANG X Y,REN S Q,et al.Deep residual learning for image recognition[C]//Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition.Los Ala-mitos:IEEE Computer Society Press,2016:770-778.
[23]KRIZHEVSKY A,SUTSKEVER I,HINTON G E.ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 26th Annual Conference on Neural Information Processing Systems.Cambridge:MIT Press,2012:1097-1105.
[24]CAO Z,LONG M,WANG J,et al.HashNet:deep learning to hash by continuation[C]//Proceedings of the IEEE Internatio-nal Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2017:5609-5618.
[25]VEDALDI A,LENC K.MatConvNet:convolutional neural net-works for MATLAB[C]//Proceedings of the 23rd ACM International Conference on Multimedia.New York:Association for Computing Machinery,2015:689-692.
[26]LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-basedlearning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[27]KRIZHEVSKY A,HINTON G.Learning multiple layers of features from tiny images[D].Toronto:University of Toronto,2009.
[28]CHUA T S,TANG J,HONG R,et al.NUS-WIDE:a real-world web image database from national university of singapore[C]//Proceedings of the ACM International Conference on Image and Video Retrieval.New York:Association for Computing Machi-nery,2009:1-9.
[29]CAO Y,LIU B,LONG M S,et al.HashGAN:deep learning to hash with pair conditional wasserstein GAN [C]//Proceedings of the 31st IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2018:1287-1296.

相关文章 15

[1]	朱承璋, 黄嘉儿, 肖亚龙, 王晗, 邹北骥. 基于注意力机制的医学影像深度哈希检索算法 Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism 计算机科学, 2022, 49(8): 113-119. https://doi.org/10.11896/jsjkx.210700153
[2]	苗壮, 王亚鹏, 李阳, 王家宝, 张睿, 赵昕昕. 一种鲁棒的双教师自监督蒸馏哈希学习方法 Robust Hash Learning Method Based on Dual-teacher Self-supervised Distillation 计算机科学, 2022, 49(10): 159-168. https://doi.org/10.11896/jsjkx.210800050
[3]	王春静, 刘丽, 谭艳艳, 张化祥. 基于模糊颜色特征和模糊相似度的图像检索方法 Image Retrieval Method Based on Fuzzy Color Features and Fuzzy Smiliarity 计算机科学, 2021, 48(8): 191-199. https://doi.org/10.11896/jsjkx.200800202
[4]	王晓飞, 周超, 刘利刚. 基于特征聚类的轻量级图像搜索系统 Lightweight Image Retrieval System Based on Feature Clustering 计算机科学, 2021, 48(2): 148-152. https://doi.org/10.11896/jsjkx.191200104
[5]	邹傲, 郝文宁, 靳大尉, 陈刚, 田媛. 基于预训练和深度哈希的大规模文本检索研究 Study on Text Retrieval Based on Pre-training and Deep Hash 计算机科学, 2021, 48(11): 300-306. https://doi.org/10.11896/jsjkx.210300266
[6]	徐行, 孙嘉良, 汪政, 杨阳. 基于特征变换的图像检索对抗防御 Feature Transformation for Defending Adversarial Attack on Image Retrieval 计算机科学, 2021, 48(10): 258-265. https://doi.org/10.11896/jsjkx.200800222
[7]	李金霞, 赵志刚, 李强, 吕慧显, 李明生. 改进的局部和相似性保持特征选择算法 Improved Locality and Similarity Preserving Feature Selection Algorithm 计算机科学, 2020, 47(6A): 480-484. https://doi.org/10.11896/JsJkx.20190800095
[8]	邓一姣, 张凤荔, 陈学勤, 艾擎, 余苏喆. 面向跨模态检索的协同注意力网络模型 Collaborative Attention Network Model for Cross-modal Retrieval 计算机科学, 2020, 47(4): 54-59. https://doi.org/10.11896/jsjkx.190600181
[9]	焦扬, 杨传颖, 石宝. 基于SVM相关反馈的鞋印图像检索算法 Relevance Feedback Method Based on SVM in Shoeprint Images Retrieval 计算机科学, 2020, 47(11A): 244-247. https://doi.org/10.11896/jsjkx.200400032
[10]	孙伟, 赵玉普. 增强旋转不变LBP算法及其在图像检索中的应用 Enhanced Rotation Invariant LBP Algorithm and Its Application in Image Retrieval 计算机科学, 2019, 46(7): 263-267. https://doi.org/10.11896/j.issn.1002-137X.2019.07.040
[11]	彭金喜, 苏远歧, 薛笑荣. 基于深度学习和同生矩阵的SAR图像纹理特征检索方法 SAR Image Feature Retrieval Method Based on Deep Learning and Synchronic Matrix 计算机科学, 2019, 46(6A): 196-199.
[12]	侯媛媛, 何儒汉, 李敏, 陈佳. 结合卷积神经网络多层特征融合和K-Means聚类的服装图像检索方法 Clothing Image Retrieval Method Combining Convolutional Neural Network Multi-layerFeature Fusion and K-Means Clustering 计算机科学, 2019, 46(6A): 215-221.
[13]	曾燕, 陈岳林, 蔡晓东. 一种基于权重哈希化的深度人脸识别算法 Deep Face Recognition Algorithm Based on Weighted Hashing 计算机科学, 2019, 46(6): 277-281. https://doi.org/10.11896/j.issn.1002-137X.2019.06.041
[14]	何霞, 汤一平, 王丽冉, 陈朋, 袁公萍. 基于Faster RCNNH的多任务分层图像检索技术 Multitask Hierarchical Image Retrieval Technology Based on Faster RCNNH 计算机科学, 2019, 46(3): 303-313. https://doi.org/10.11896/j.issn.1002-137X.2019.03.045
[15]	李晓雨, 聂秀山, 崔超然, 蹇木伟, 尹义龙. 基于迁移学习的图像检索算法 Image Retrieval Algorithm Based on Transfer Learning 计算机科学, 2019, 46(1): 73-77. https://doi.org/10.11896／j.issn.1002-137X.2019.01.011

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed