计算机科学 ›› 2019, Vol. 46 ›› Issue (1): 73-77.doi: 10.11896/j.issn.1002-137X.2019.01.011

• 2018 年第七届中国数据挖掘会议 • 上一篇    下一篇

基于迁移学习的图像检索算法

李晓雨1, 聂秀山1, 崔超然1, 蹇木伟1, 尹义龙2   

  1. (山东财经大学计算机科学与技术学院 济南250014)1
    (山东大学软件学院 济南250014)2
  • 收稿日期:2018-05-08 出版日期:2019-01-15 发布日期:2019-02-25
  • 作者简介:李晓雨(1994-),女,硕士生,主要研究方向为机器学习、多媒体信息处理;聂秀山(1981-),博士,教授,主要研究方向为数据挖掘、多媒体信息检索和机器视觉,E-mail:niexiushan@163.com(通信作者);崔超然(1987-),博士,教授,主要研究方向为信息检索、推荐系统和机器学习;蹇木伟(1982-),博士,教授,主要研究方向为人脸识别、图像视频处理、机器学习和机器视觉;尹义龙(1972-),博士,教授,主要研究方向为机器学习、数据挖掘和计算机医学。
  • 基金资助:
    山东高等学校科技计划项目(JI7KB161),国家自然科学基金(61671274),中国博士后基金(2016M592190),山东省高等学校优势学科人才团队培育计划,山东财经大学研究生教育创新计划(SCY1604)资助

Image Retrieval Algorithm Based on Transfer Learning

LI Xiao-yu1, NIE Xiu-shan1, CUI Chao-ran1, JIAN Mu-wei1, YIN Yi-long2   

  1. (School of Computer Science and Technology,Shandong University of Finance and Economics,Jinan 250014,China)1
    (School of Software,Shandong University,Jinan 250014,China)2
  • Received:2018-05-08 Online:2019-01-15 Published:2019-02-25

摘要: 近年来,随着互联网的发展和智能设备的普及,网络上存储的图片数量呈现爆发式增长,同时,不同类型的社交网络、媒体的用户数量也连续增长。在这种情况下,网络上的多媒体数据类型也发生了变革,在包含其本身携带的视觉信息的同时,也包含用户为其设定的标签信息、文本信息。在这种多模态信息杂糅的环境下,如何向用户提供快速准确的图像检索结果,是多媒体检索领域的一个新挑战。文中提出了一种基于迁移学习的图像检索算法,在对图像的视觉信息进行学习的同时,也对图像的文本信息进行学习,并将学习到的结果迁移到视觉信息领域,进行跨模态信息融合,进而产生包含跨模态信息的图像特征。经实验证明,所提算法能够实现更优的图像检索结果。

关键词: 图像检索, 跨模态, 迁移学习, 特征提取

Abstract: In recent years,with the development of the Internet and the popularity of smart devices,the number of online store image is explosively growing.At the same time,the number of users who use different types of social networks and media continues to grow.In this case,the multimedia data type that the user uploaded to the network also has changed,the image uploaded by the user contains the visual information that is carried by the image itself,and also contains the label information and text information that the user sets for it.Therefore,how to provide fast and accurate image retrieval results to users is a new challenge in the field of multimedia retrieval.This paper proposed an image retrieval algorithm based on transfer learning.It learns the visual information and the text information at the same time,then migrates the results learnt to the visual information domain,and thus the feature contains cross modal information.Experimental results show that the proposed algorithm can achieve better image retrieval results.

Key words: Image retrieval, Cross-modal, Transfer learning, Feature extraction

中图分类号: 

  • TP391
[1]SCHNEIDER M,SHIHFU C.A Robust Content Based DigitalSignature for Image Authentication[C]//Proceedings,International Conference on Image Processing.1996:227-230.<br /> [2]JIN Y.Image feature extraction algorithm based on PCA/ICA[D].Xi’an:Xi’an University of Electronic Science and Techno-logy,2014.(in Chinese)<br /> 靳洋.基于PCA/ICA的图像特征提取算法研究[D].西安:西安电子科技大学,2014.<br /> [3]WANG E Y.Study on the extraction and recognition of gray image features based on fuzzy clustering[D].Kunming:Yunnan University,2010.(in Chinese)<br /> 王恩永.基于模糊聚类的灰度图像特征提取和识别研究[D].昆明:云南大学,2010.<br /> [4]ZHANG Z L,LI J C,SHEN Z K.On texture feature extraction based on local Walsh transform[J].The signal processing,2005,21(6):589-596.(in Chinese)<br /> 张志龙,李吉成,沈振康.基于局部沃尔什变换的纹理特征提取方法研究[J].信号处理,2005,21(6):589-596.<br /> [5]SATPATHY A,JIANG X,ENG H L.LBP-Based Edge-Texture Features for Object Recognition [J].IEEE Transactions on Ima-ge Processing,2014,23(5):1953-1964.<br /> [6]KIRBY M,SIROVICH L.Application of the Karhunen-Loeve Procedure for the Characterization of Human Faces[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2002,12(1):103-108.<br /> [7]BELL A J,SEJNOWSKI T J.The independent components of natural scenes are edge filters[J].Vision Research,1997,37(23):3327-3338.<br /> [8]DENG L P.A study of SVM algorithm for face images under multiple algebraic feature extraction methods[J].Information security and Technology,2014,5(10):45-47.(in Chinese)<br /> 邓丽萍.多种代数特征抽取方法下的人脸图像SVM算法研究[J].信息安全与技术,2014,5(10):45-47.<br /> [9]URVOY M,GOUDIA D,AUTRUSSEAU F.Perceptual DFT Watermarking With Improved Detection and Robustness to Geometrical Distortions[J].IEEE Transactions on Information Forensics & Security,2014,9(7):1108-1119.<br /> [10]ZORAN M,ZORAN V.Robustness of SVD Watermarks in Video Sequences Encoded with H.264/AVC[C]//International Scientific Conference on Information,Communication and Energy Systems and Technologies.2014.<br /> [11]HU T S,ZHOU W,JIANG C C.A method of face recognition based on DCT coefficient and Fourier descriptor[J].Journal of Zhejiang University of Technology,2010,38(5):557-560.(in Chinese)<br /> 胡同森,周维,蒋成成.一种基于DCT系数和Fourier描述子的人脸识别方法[J].浙江工业大学学报,2010,38(5):557-560.<br /> [12]LOWE D G.Distinctive Image Features from Scale-Invariant Keypoints[J].International Journal of Computer Vision,2004,60(2):91-110.<br /> [13]BLEI D M,NG A Y,JORDAN M I.Latent dirichlet allocation[J].Journal of Machine Learning Research,2003,3:993-1022.<br /> [14]QUATTONI A,COLLINS M,DARRELL T.Transfer learning for image classification with sparse prototype representations[C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2008:1-8.<br /> [15]PAN S J,YANG Q.A Survey on Transfer Learning[J].IEEE Transactions on Knowledge & Data Engineering,2010,22(10):1345-1359.<br /> [16]OQUAB M,BOTTOU L,LAPTEV I,et al.Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks[C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2014:1717-1724.<br /> [17]TAYLOR M E,STONE P.Transfer Learning for Reinforcement Learning Domains:A Survey[J].Journal of Machine Learning Research,2009,10(10):1633-1685.<br /> [18]DAI W,JIN O,XUE G R,et al.EigenTransfer:a unified framework for transfer learning[C]//International Conference on Machine Learning.ACM,2009:193-200.<br /> [19]ROY S D,MEI T,ZENG W,et al.Social Transfer:cross-domain transfer learning from social streams for media applications[C]//ACM International Conference on Multimedia.ACM,2012:649-658.<br /> [20]TAHMORESNEZHAD J,HASHEMI S.Visual domain adaptation via transfer feature learning[J].Knowledge & Information Systems,2016,50(2):1-21.<br /> [21]NIE W,LIU A,SU Y.Cross-domain semantic transfer from large-scale social media[J].Multimedia Systems,2016,22(1):75-85.<br /> [22]SHAO L,ZHU F,LI X.Transfer Learning for Visual Categorization:A Survey[J].IEEE Transactions on Neural Networks & Learning Systems,2015,26(5):1019-1034.<br /> [23]TAYLOR M E,STONE P.Transfer Learning for Reinforcement Learning Domains:A Survey[J].Journal of Machine Learning Research,2009,10(10):1633-1685.<br /> [24]ZHU F,SHAO L.Weakly-Supervised Cross-Domain Dictionary Learning for Visual Recognition[J].International Journal of Computer Vision,2014,109(1-2):42-59.<br /> [25]LI X,ZHANG L,DU B,et al.Iterative Reweighting Heterogeneous Transfer Learning Framework for Supervised Remote Sensing Image Classification[J].IEEE Journal of Selected To-pics in Applied Earth Observations & Remote Sensing,2017,10(5):2022-2035.<br /> [26]DING Z,FU Y.Robust Transfer Metric Learning for Image Classification[J].IEEE Transactions on Image Processing,2017,PP(99):1.<br /> [27]GHAZI M M,YANIKOGLU B,APTOULA E.Plant identification using deep neural networks via optimization of transfer learning parameters[J].Neurocomputing,2017,235:228-235.<br /> [28]SHI Z,SIVA P,XIANG T.Transfer Learning by Ranking for Weakly Supervised Object Annotation[OL].http://www.bmva.org/bmvc.2012/BMVC/paper078/abstract078.pdf.<br /> [29]RAVISHANKAR H,SUDHAKAR P,VENKATARAMANI R,et al.Understanding the Mechanisms of Deep Transfer Learning for Medical Images[C]//International Workshop on Large-scale Annotation of Biomedical Data & Expert Lablel Synthesis.2016:188-196.
[1] 刘洋, 金忠. 一种结合非局部和多区域注意力机制的细粒度图像识别方法[J]. 计算机科学, 2021, 48(1): 197-203.
[2] 欧阳鹏, 陆璐, 张凡龙, 邱少健. 基于迁移学习和过采样技术的跨项目克隆代码一致性维护需求预测[J]. 计算机科学, 2020, 47(9): 10-16.
[3] 暴雨轩, 芦天亮, 杜彦辉. 深度伪造视频检测技术综述[J]. 计算机科学, 2020, 47(9): 283-292.
[4] 汪亮, 周新志, 严华. 基于GPU的实时SIFT算法[J]. 计算机科学, 2020, 47(8): 105-111.
[5] 袁晨晖, 程春玲. 基于PE散度实例过滤的深度域适应方法[J]. 计算机科学, 2020, 47(8): 151-156.
[6] 梁正友, 何景琳, 孙宇. 一种用于微表情自动识别的三维卷积神经网络进化方法[J]. 计算机科学, 2020, 47(8): 227-232.
[7] 罗婷瑞, 贾建, 张瑞. 基于可调Q因子小波变换和迁移学习的癫痫脑电信号检测[J]. 计算机科学, 2020, 47(7): 199-205.
[8] 杨威超, 郭渊博, 李涛, 朱本全. 基于流量指纹的物联网设备识别方法和物联网安全模型[J]. 计算机科学, 2020, 47(7): 299-306.
[9] 蓝章礼, 申德兴, 曹娟, 张玉欣. 一种基图像提取和内容无关图像重构方法研究[J]. 计算机科学, 2020, 47(6A): 226-229.
[10] 周立鹏, 孟利民, 周磊, 蒋维, 董建平. 基于BP神经网络的摔倒检测算法[J]. 计算机科学, 2020, 47(6A): 242-246.
[11] 袁得嵛, 章逸钒, 高见, 孙海春. 基于用户特征提取的新浪微博异常用户检测方法[J]. 计算机科学, 2020, 47(6A): 364-368.
[12] 郑纯军, 王春立, 贾宁. 语音任务下声学特征提取综述[J]. 计算机科学, 2020, 47(5): 110-119.
[13] 邓一姣, 张凤荔, 陈学勤, 艾擎, 余苏喆. 面向跨模态检索的协同注意力网络模型[J]. 计算机科学, 2020, 47(4): 54-59.
[14] 赵楠, 皮文超, 许长桥. 一种面向多维特征分析过滤的视频推荐算法[J]. 计算机科学, 2020, 47(4): 103-107.
[15] 王昆仑, 刘文璨, 何小海, 卿粼波, 吴晓红. 一种用于异常行为检测的运动特征描述子[J]. 计算机科学, 2020, 47(4): 119-124.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 编辑部. 新网站开通,欢迎大家订阅![J]. 计算机科学, 2018, 1(1): 1 .
[2] 雷丽晖,王静. 可能性测度下的LTL模型检测并行化研究[J]. 计算机科学, 2018, 45(4): 71 -75 .
[3] 孙启,金燕,何琨,徐凌轩. 用于求解混合车辆路径问题的混合进化算法[J]. 计算机科学, 2018, 45(4): 76 -82 .
[4] 张佳男,肖鸣宇. 带权混合支配问题的近似算法研究[J]. 计算机科学, 2018, 45(4): 83 -88 .
[5] 伍建辉,黄中祥,李武,吴健辉,彭鑫,张生. 城市道路建设时序决策的鲁棒优化[J]. 计算机科学, 2018, 45(4): 89 -93 .
[6] 史雯隽,武继刚,罗裕春. 针对移动云计算任务迁移的快速高效调度算法[J]. 计算机科学, 2018, 45(4): 94 -99 .
[7] 周燕萍,业巧林. 基于L1-范数距离的最小二乘对支持向量机[J]. 计算机科学, 2018, 45(4): 100 -105 .
[8] 刘博艺,唐湘滟,程杰仁. 基于多生长时期模板匹配的玉米螟识别方法[J]. 计算机科学, 2018, 45(4): 106 -111 .
[9] 耿海军,施新刚,王之梁,尹霞,尹少平. 基于有向无环图的互联网域内节能路由算法[J]. 计算机科学, 2018, 45(4): 112 -116 .
[10] 崔琼,李建华,王宏,南明莉. 基于节点修复的网络化指挥信息系统弹性分析模型[J]. 计算机科学, 2018, 45(4): 117 -121 .