计算机科学 ›› 2019, Vol. 46 ›› Issue (1): 73-77.doi: 10.11896/j.issn.1002-137X.2019.01.011

• 2018 年第七届中国数据挖掘会议 • 上一篇    下一篇

基于迁移学习的图像检索算法

李晓雨1, 聂秀山1, 崔超然1, 蹇木伟1, 尹义龙2   

  1. (山东财经大学计算机科学与技术学院 济南250014)1
    (山东大学软件学院 济南250014)2
  • 收稿日期:2018-05-08 出版日期:2019-01-15 发布日期:2019-02-25
  • 作者简介:李晓雨(1994-),女,硕士生,主要研究方向为机器学习、多媒体信息处理;聂秀山(1981-),博士,教授,主要研究方向为数据挖掘、多媒体信息检索和机器视觉,E-mail:niexiushan@163.com(通信作者);崔超然(1987-),博士,教授,主要研究方向为信息检索、推荐系统和机器学习;蹇木伟(1982-),博士,教授,主要研究方向为人脸识别、图像视频处理、机器学习和机器视觉;尹义龙(1972-),博士,教授,主要研究方向为机器学习、数据挖掘和计算机医学。
  • 基金资助:
    山东高等学校科技计划项目(JI7KB161),国家自然科学基金(61671274),中国博士后基金(2016M592190),山东省高等学校优势学科人才团队培育计划,山东财经大学研究生教育创新计划(SCY1604)资助

Image Retrieval Algorithm Based on Transfer Learning

LI Xiao-yu1, NIE Xiu-shan1, CUI Chao-ran1, JIAN Mu-wei1, YIN Yi-long2   

  1. (School of Computer Science and Technology,Shandong University of Finance and Economics,Jinan 250014,China)1
    (School of Software,Shandong University,Jinan 250014,China)2
  • Received:2018-05-08 Online:2019-01-15 Published:2019-02-25

摘要: 近年来,随着互联网的发展和智能设备的普及,网络上存储的图片数量呈现爆发式增长,同时,不同类型的社交网络、媒体的用户数量也连续增长。在这种情况下,网络上的多媒体数据类型也发生了变革,在包含其本身携带的视觉信息的同时,也包含用户为其设定的标签信息、文本信息。在这种多模态信息杂糅的环境下,如何向用户提供快速准确的图像检索结果,是多媒体检索领域的一个新挑战。文中提出了一种基于迁移学习的图像检索算法,在对图像的视觉信息进行学习的同时,也对图像的文本信息进行学习,并将学习到的结果迁移到视觉信息领域,进行跨模态信息融合,进而产生包含跨模态信息的图像特征。经实验证明,所提算法能够实现更优的图像检索结果。

关键词: 图像检索, 跨模态, 迁移学习, 特征提取

Abstract: In recent years,with the development of the Internet and the popularity of smart devices,the number of online store image is explosively growing.At the same time,the number of users who use different types of social networks and media continues to grow.In this case,the multimedia data type that the user uploaded to the network also has changed,the image uploaded by the user contains the visual information that is carried by the image itself,and also contains the label information and text information that the user sets for it.Therefore,how to provide fast and accurate image retrieval results to users is a new challenge in the field of multimedia retrieval.This paper proposed an image retrieval algorithm based on transfer learning.It learns the visual information and the text information at the same time,then migrates the results learnt to the visual information domain,and thus the feature contains cross modal information.Experimental results show that the proposed algorithm can achieve better image retrieval results.

Key words: Image retrieval, Cross-modal, Transfer learning, Feature extraction

中图分类号: 

  • TP391
[1]SCHNEIDER M,SHIHFU C.A Robust Content Based DigitalSignature for Image Authentication[C]//Proceedings,International Conference on Image Processing.1996:227-230.<br /> [2]JIN Y.Image feature extraction algorithm based on PCA/ICA[D].Xi’an:Xi’an University of Electronic Science and Techno-logy,2014.(in Chinese)<br /> 靳洋.基于PCA/ICA的图像特征提取算法研究[D].西安:西安电子科技大学,2014.<br /> [3]WANG E Y.Study on the extraction and recognition of gray image features based on fuzzy clustering[D].Kunming:Yunnan University,2010.(in Chinese)<br /> 王恩永.基于模糊聚类的灰度图像特征提取和识别研究[D].昆明:云南大学,2010.<br /> [4]ZHANG Z L,LI J C,SHEN Z K.On texture feature extraction based on local Walsh transform[J].The signal processing,2005,21(6):589-596.(in Chinese)<br /> 张志龙,李吉成,沈振康.基于局部沃尔什变换的纹理特征提取方法研究[J].信号处理,2005,21(6):589-596.<br /> [5]SATPATHY A,JIANG X,ENG H L.LBP-Based Edge-Texture Features for Object Recognition [J].IEEE Transactions on Ima-ge Processing,2014,23(5):1953-1964.<br /> [6]KIRBY M,SIROVICH L.Application of the Karhunen-Loeve Procedure for the Characterization of Human Faces[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2002,12(1):103-108.<br /> [7]BELL A J,SEJNOWSKI T J.The independent components of natural scenes are edge filters[J].Vision Research,1997,37(23):3327-3338.<br /> [8]DENG L P.A study of SVM algorithm for face images under multiple algebraic feature extraction methods[J].Information security and Technology,2014,5(10):45-47.(in Chinese)<br /> 邓丽萍.多种代数特征抽取方法下的人脸图像SVM算法研究[J].信息安全与技术,2014,5(10):45-47.<br /> [9]URVOY M,GOUDIA D,AUTRUSSEAU F.Perceptual DFT Watermarking With Improved Detection and Robustness to Geometrical Distortions[J].IEEE Transactions on Information Forensics & Security,2014,9(7):1108-1119.<br /> [10]ZORAN M,ZORAN V.Robustness of SVD Watermarks in Video Sequences Encoded with H.264/AVC[C]//International Scientific Conference on Information,Communication and Energy Systems and Technologies.2014.<br /> [11]HU T S,ZHOU W,JIANG C C.A method of face recognition based on DCT coefficient and Fourier descriptor[J].Journal of Zhejiang University of Technology,2010,38(5):557-560.(in Chinese)<br /> 胡同森,周维,蒋成成.一种基于DCT系数和Fourier描述子的人脸识别方法[J].浙江工业大学学报,2010,38(5):557-560.<br /> [12]LOWE D G.Distinctive Image Features from Scale-Invariant Keypoints[J].International Journal of Computer Vision,2004,60(2):91-110.<br /> [13]BLEI D M,NG A Y,JORDAN M I.Latent dirichlet allocation[J].Journal of Machine Learning Research,2003,3:993-1022.<br /> [14]QUATTONI A,COLLINS M,DARRELL T.Transfer learning for image classification with sparse prototype representations[C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2008:1-8.<br /> [15]PAN S J,YANG Q.A Survey on Transfer Learning[J].IEEE Transactions on Knowledge & Data Engineering,2010,22(10):1345-1359.<br /> [16]OQUAB M,BOTTOU L,LAPTEV I,et al.Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks[C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2014:1717-1724.<br /> [17]TAYLOR M E,STONE P.Transfer Learning for Reinforcement Learning Domains:A Survey[J].Journal of Machine Learning Research,2009,10(10):1633-1685.<br /> [18]DAI W,JIN O,XUE G R,et al.EigenTransfer:a unified framework for transfer learning[C]//International Conference on Machine Learning.ACM,2009:193-200.<br /> [19]ROY S D,MEI T,ZENG W,et al.Social Transfer:cross-domain transfer learning from social streams for media applications[C]//ACM International Conference on Multimedia.ACM,2012:649-658.<br /> [20]TAHMORESNEZHAD J,HASHEMI S.Visual domain adaptation via transfer feature learning[J].Knowledge & Information Systems,2016,50(2):1-21.<br /> [21]NIE W,LIU A,SU Y.Cross-domain semantic transfer from large-scale social media[J].Multimedia Systems,2016,22(1):75-85.<br /> [22]SHAO L,ZHU F,LI X.Transfer Learning for Visual Categorization:A Survey[J].IEEE Transactions on Neural Networks & Learning Systems,2015,26(5):1019-1034.<br /> [23]TAYLOR M E,STONE P.Transfer Learning for Reinforcement Learning Domains:A Survey[J].Journal of Machine Learning Research,2009,10(10):1633-1685.<br /> [24]ZHU F,SHAO L.Weakly-Supervised Cross-Domain Dictionary Learning for Visual Recognition[J].International Journal of Computer Vision,2014,109(1-2):42-59.<br /> [25]LI X,ZHANG L,DU B,et al.Iterative Reweighting Heterogeneous Transfer Learning Framework for Supervised Remote Sensing Image Classification[J].IEEE Journal of Selected To-pics in Applied Earth Observations & Remote Sensing,2017,10(5):2022-2035.<br /> [26]DING Z,FU Y.Robust Transfer Metric Learning for Image Classification[J].IEEE Transactions on Image Processing,2017,PP(99):1.<br /> [27]GHAZI M M,YANIKOGLU B,APTOULA E.Plant identification using deep neural networks via optimization of transfer learning parameters[J].Neurocomputing,2017,235:228-235.<br /> [28]SHI Z,SIVA P,XIANG T.Transfer Learning by Ranking for Weakly Supervised Object Annotation[OL].http://www.bmva.org/bmvc.2012/BMVC/paper078/abstract078.pdf.<br /> [29]RAVISHANKAR H,SUDHAKAR P,VENKATARAMANI R,et al.Understanding the Mechanisms of Deep Transfer Learning for Medical Images[C]//International Workshop on Large-scale Annotation of Biomedical Data & Expert Lablel Synthesis.2016:188-196.
[1] 周燕, 曾凡智, 吴臣, 罗粤, 刘紫琴. 基于深度学习的三维形状特征提取方法[J]. 计算机科学, 2019, 46(9): 47-58.
[2] 杜臻, 马立鹏, 孙国梓. 一种基于小波分析的网络流量异常检测方法[J]. 计算机科学, 2019, 46(8): 178-182.
[3] 折蓉蓉, 张丽萍. 基于软件演化历史识别并推荐重构克隆的方法[J]. 计算机科学, 2019, 46(8): 224-232.
[4] 韩慧,王黎明,柴玉梅,刘箴. 基于强化表征学习深度森林的文本情感分类[J]. 计算机科学, 2019, 46(7): 172-179.
[5] 孙伟, 赵玉普. 增强旋转不变LBP算法及其在图像检索中的应用[J]. 计算机科学, 2019, 46(7): 263-267.
[6] 彭金喜, 苏远歧, 薛笑荣. 基于深度学习和同生矩阵的SAR图像纹理特征检索方法[J]. 计算机科学, 2019, 46(6A): 196-199.
[7] 侯媛媛, 何儒汉, 李敏, 陈佳. 结合卷积神经网络多层特征融合和K-Means聚类的服装图像检索方法[J]. 计算机科学, 2019, 46(6A): 215-221.
[8] 李月峰. 一种基于多特征结合的三维模型检索方法[J]. 计算机科学, 2019, 46(6A): 266-269.
[9] 韩笑, 张晶, 李月龙. 基于手势几何分布特征的手势识别[J]. 计算机科学, 2019, 46(6A): 246-249.
[10] 何孝文, 胡一飞, 王海平, 陈默. 在线学习非负矩阵分解[J]. 计算机科学, 2019, 46(6A): 473-477.
[11] 周彬彬, 张宏军, 张睿, 冯蕴天, 徐有为. 面向实体标注的军事语料库建设[J]. 计算机科学, 2019, 46(6A): 540-546.
[12] 徐一鸣, 张娟, 刘成成, 顾菊平, 潘高超. 迁移学习模式下基于GoogLeNet网络的风电机组视觉检测[J]. 计算机科学, 2019, 46(5): 260-265.
[13] 许磊, 王建新. 基于模糊神经网络的异常网络数据挖掘算法[J]. 计算机科学, 2019, 46(4): 73-76.
[14] 刘晓虹, 朱玉全, 刘哲, 宋余庆, 朱, 彦, 袁德琪. 基于改进多尺度LBP算法的肝脏CT图像特征提取方法[J]. 计算机科学, 2019, 46(3): 125-130.
[15] 李荫民, 薛凯心, 高赞, 薛彦兵, 徐光平, 张桦. 基于残差网络的三维模型检索算法[J]. 计算机科学, 2019, 46(3): 148-153.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 编辑部. 新网站开通,欢迎大家订阅![J]. 计算机科学, 2018, 1(1): 1 .
[2] 雷丽晖,王静. 可能性测度下的LTL模型检测并行化研究[J]. 计算机科学, 2018, 45(4): 71 -75, 88 .
[3] 夏庆勋,庄毅. 一种基于局部性原理的远程验证机制[J]. 计算机科学, 2018, 45(4): 148 -151, 162 .
[4] 厉柏伸,李领治,孙涌,朱艳琴. 基于伪梯度提升决策树的内网防御算法[J]. 计算机科学, 2018, 45(4): 157 -162 .
[5] 王欢,张云峰,张艳. 一种基于CFDs规则的修复序列快速判定方法[J]. 计算机科学, 2018, 45(3): 311 -316 .
[6] 孙启,金燕,何琨,徐凌轩. 用于求解混合车辆路径问题的混合进化算法[J]. 计算机科学, 2018, 45(4): 76 -82 .
[7] 张佳男,肖鸣宇. 带权混合支配问题的近似算法研究[J]. 计算机科学, 2018, 45(4): 83 -88 .
[8] 伍建辉,黄中祥,李武,吴健辉,彭鑫,张生. 城市道路建设时序决策的鲁棒优化[J]. 计算机科学, 2018, 45(4): 89 -93 .
[9] 刘琴. 计算机取证过程中基于约束的数据质量问题研究[J]. 计算机科学, 2018, 45(4): 169 -172 .
[10] 钟菲,杨斌. 基于主成分分析网络的车牌检测方法[J]. 计算机科学, 2018, 45(3): 268 -273 .