计算机科学 ›› 2019, Vol. 46 ›› Issue (1): 73-77.doi: 10.11896/j.issn.1002-137X.2019.01.011

• 2018 年第七届中国数据挖掘会议 • 上一篇    下一篇

基于迁移学习的图像检索算法

李晓雨1, 聂秀山1, 崔超然1, 蹇木伟1, 尹义龙2   

  1. (山东财经大学计算机科学与技术学院 济南250014)1
    (山东大学软件学院 济南250014)2
  • 收稿日期:2018-05-08 出版日期:2019-01-15 发布日期:2019-02-25
  • 作者简介:李晓雨(1994-),女,硕士生,主要研究方向为机器学习、多媒体信息处理;聂秀山(1981-),博士,教授,主要研究方向为数据挖掘、多媒体信息检索和机器视觉,E-mail:niexiushan@163.com(通信作者);崔超然(1987-),博士,教授,主要研究方向为信息检索、推荐系统和机器学习;蹇木伟(1982-),博士,教授,主要研究方向为人脸识别、图像视频处理、机器学习和机器视觉;尹义龙(1972-),博士,教授,主要研究方向为机器学习、数据挖掘和计算机医学。
  • 基金资助:
    山东高等学校科技计划项目(JI7KB161),国家自然科学基金(61671274),中国博士后基金(2016M592190),山东省高等学校优势学科人才团队培育计划,山东财经大学研究生教育创新计划(SCY1604)资助

Image Retrieval Algorithm Based on Transfer Learning

LI Xiao-yu1, NIE Xiu-shan1, CUI Chao-ran1, JIAN Mu-wei1, YIN Yi-long2   

  1. (School of Computer Science and Technology,Shandong University of Finance and Economics,Jinan 250014,China)1
    (School of Software,Shandong University,Jinan 250014,China)2
  • Received:2018-05-08 Online:2019-01-15 Published:2019-02-25

摘要: 近年来,随着互联网的发展和智能设备的普及,网络上存储的图片数量呈现爆发式增长,同时,不同类型的社交网络、媒体的用户数量也连续增长。在这种情况下,网络上的多媒体数据类型也发生了变革,在包含其本身携带的视觉信息的同时,也包含用户为其设定的标签信息、文本信息。在这种多模态信息杂糅的环境下,如何向用户提供快速准确的图像检索结果,是多媒体检索领域的一个新挑战。文中提出了一种基于迁移学习的图像检索算法,在对图像的视觉信息进行学习的同时,也对图像的文本信息进行学习,并将学习到的结果迁移到视觉信息领域,进行跨模态信息融合,进而产生包含跨模态信息的图像特征。经实验证明,所提算法能够实现更优的图像检索结果。

关键词: 跨模态, 迁移学习, 特征提取, 图像检索

Abstract: In recent years,with the development of the Internet and the popularity of smart devices,the number of online store image is explosively growing.At the same time,the number of users who use different types of social networks and media continues to grow.In this case,the multimedia data type that the user uploaded to the network also has changed,the image uploaded by the user contains the visual information that is carried by the image itself,and also contains the label information and text information that the user sets for it.Therefore,how to provide fast and accurate image retrieval results to users is a new challenge in the field of multimedia retrieval.This paper proposed an image retrieval algorithm based on transfer learning.It learns the visual information and the text information at the same time,then migrates the results learnt to the visual information domain,and thus the feature contains cross modal information.Experimental results show that the proposed algorithm can achieve better image retrieval results.

Key words: Cross-modal, Feature extraction, Image retrieval, Transfer learning

中图分类号: 

  • TP391
[1]SCHNEIDER M,SHIHFU C.A Robust Content Based DigitalSignature for Image Authentication[C]//Proceedings,International Conference on Image Processing.1996:227-230.<br /> [2]JIN Y.Image feature extraction algorithm based on PCA/ICA[D].Xi’an:Xi’an University of Electronic Science and Techno-logy,2014.(in Chinese)<br /> 靳洋.基于PCA/ICA的图像特征提取算法研究[D].西安:西安电子科技大学,2014.<br /> [3]WANG E Y.Study on the extraction and recognition of gray image features based on fuzzy clustering[D].Kunming:Yunnan University,2010.(in Chinese)<br /> 王恩永.基于模糊聚类的灰度图像特征提取和识别研究[D].昆明:云南大学,2010.<br /> [4]ZHANG Z L,LI J C,SHEN Z K.On texture feature extraction based on local Walsh transform[J].The signal processing,2005,21(6):589-596.(in Chinese)<br /> 张志龙,李吉成,沈振康.基于局部沃尔什变换的纹理特征提取方法研究[J].信号处理,2005,21(6):589-596.<br /> [5]SATPATHY A,JIANG X,ENG H L.LBP-Based Edge-Texture Features for Object Recognition [J].IEEE Transactions on Ima-ge Processing,2014,23(5):1953-1964.<br /> [6]KIRBY M,SIROVICH L.Application of the Karhunen-Loeve Procedure for the Characterization of Human Faces[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2002,12(1):103-108.<br /> [7]BELL A J,SEJNOWSKI T J.The independent components of natural scenes are edge filters[J].Vision Research,1997,37(23):3327-3338.<br /> [8]DENG L P.A study of SVM algorithm for face images under multiple algebraic feature extraction methods[J].Information security and Technology,2014,5(10):45-47.(in Chinese)<br /> 邓丽萍.多种代数特征抽取方法下的人脸图像SVM算法研究[J].信息安全与技术,2014,5(10):45-47.<br /> [9]URVOY M,GOUDIA D,AUTRUSSEAU F.Perceptual DFT Watermarking With Improved Detection and Robustness to Geometrical Distortions[J].IEEE Transactions on Information Forensics & Security,2014,9(7):1108-1119.<br /> [10]ZORAN M,ZORAN V.Robustness of SVD Watermarks in Video Sequences Encoded with H.264/AVC[C]//International Scientific Conference on Information,Communication and Energy Systems and Technologies.2014.<br /> [11]HU T S,ZHOU W,JIANG C C.A method of face recognition based on DCT coefficient and Fourier descriptor[J].Journal of Zhejiang University of Technology,2010,38(5):557-560.(in Chinese)<br /> 胡同森,周维,蒋成成.一种基于DCT系数和Fourier描述子的人脸识别方法[J].浙江工业大学学报,2010,38(5):557-560.<br /> [12]LOWE D G.Distinctive Image Features from Scale-Invariant Keypoints[J].International Journal of Computer Vision,2004,60(2):91-110.<br /> [13]BLEI D M,NG A Y,JORDAN M I.Latent dirichlet allocation[J].Journal of Machine Learning Research,2003,3:993-1022.<br /> [14]QUATTONI A,COLLINS M,DARRELL T.Transfer learning for image classification with sparse prototype representations[C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2008:1-8.<br /> [15]PAN S J,YANG Q.A Survey on Transfer Learning[J].IEEE Transactions on Knowledge & Data Engineering,2010,22(10):1345-1359.<br /> [16]OQUAB M,BOTTOU L,LAPTEV I,et al.Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks[C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2014:1717-1724.<br /> [17]TAYLOR M E,STONE P.Transfer Learning for Reinforcement Learning Domains:A Survey[J].Journal of Machine Learning Research,2009,10(10):1633-1685.<br /> [18]DAI W,JIN O,XUE G R,et al.EigenTransfer:a unified framework for transfer learning[C]//International Conference on Machine Learning.ACM,2009:193-200.<br /> [19]ROY S D,MEI T,ZENG W,et al.Social Transfer:cross-domain transfer learning from social streams for media applications[C]//ACM International Conference on Multimedia.ACM,2012:649-658.<br /> [20]TAHMORESNEZHAD J,HASHEMI S.Visual domain adaptation via transfer feature learning[J].Knowledge & Information Systems,2016,50(2):1-21.<br /> [21]NIE W,LIU A,SU Y.Cross-domain semantic transfer from large-scale social media[J].Multimedia Systems,2016,22(1):75-85.<br /> [22]SHAO L,ZHU F,LI X.Transfer Learning for Visual Categorization:A Survey[J].IEEE Transactions on Neural Networks & Learning Systems,2015,26(5):1019-1034.<br /> [23]TAYLOR M E,STONE P.Transfer Learning for Reinforcement Learning Domains:A Survey[J].Journal of Machine Learning Research,2009,10(10):1633-1685.<br /> [24]ZHU F,SHAO L.Weakly-Supervised Cross-Domain Dictionary Learning for Visual Recognition[J].International Journal of Computer Vision,2014,109(1-2):42-59.<br /> [25]LI X,ZHANG L,DU B,et al.Iterative Reweighting Heterogeneous Transfer Learning Framework for Supervised Remote Sensing Image Classification[J].IEEE Journal of Selected To-pics in Applied Earth Observations & Remote Sensing,2017,10(5):2022-2035.<br /> [26]DING Z,FU Y.Robust Transfer Metric Learning for Image Classification[J].IEEE Transactions on Image Processing,2017,PP(99):1.<br /> [27]GHAZI M M,YANIKOGLU B,APTOULA E.Plant identification using deep neural networks via optimization of transfer learning parameters[J].Neurocomputing,2017,235:228-235.<br /> [28]SHI Z,SIVA P,XIANG T.Transfer Learning by Ranking for Weakly Supervised Object Annotation[OL].http://www.bmva.org/bmvc.2012/BMVC/paper078/abstract078.pdf.<br /> [29]RAVISHANKAR H,SUDHAKAR P,VENKATARAMANI R,et al.Understanding the Mechanisms of Deep Transfer Learning for Medical Images[C]//International Workshop on Large-scale Annotation of Biomedical Data & Expert Lablel Synthesis.2016:188-196.
[1] 聂秀山, 潘嘉男, 谭智方, 刘新放, 郭杰, 尹义龙.
基于自然语言的视频片段定位综述
Overview of Natural Language Video Localization
计算机科学, 2022, 49(9): 111-122. https://doi.org/10.11896/jsjkx.220500130
[2] 方义秋, 张震坤, 葛君伟.
基于自注意力机制和迁移学习的跨领域推荐算法
Cross-domain Recommendation Algorithm Based on Self-attention Mechanism and Transfer Learning
计算机科学, 2022, 49(8): 70-77. https://doi.org/10.11896/jsjkx.210600011
[3] 张源, 康乐, 宫朝辉, 张志鸿.
基于Bi-LSTM的期货市场关联交易行为检测方法
Related Transaction Behavior Detection in Futures Market Based on Bi-LSTM
计算机科学, 2022, 49(7): 31-39. https://doi.org/10.11896/jsjkx.210400304
[4] 曾志贤, 曹建军, 翁年凤, 蒋国权, 徐滨.
基于注意力机制的细粒度语义关联视频-文本跨模态实体分辨
Fine-grained Semantic Association Video-Text Cross-modal Entity Resolution Based on Attention Mechanism
计算机科学, 2022, 49(7): 106-112. https://doi.org/10.11896/jsjkx.210500224
[5] 程成, 降爱莲.
基于多路径特征提取的实时语义分割方法
Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction
计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[6] 刘伟业, 鲁慧民, 李玉鹏, 马宁.
指静脉识别技术研究综述
Survey on Finger Vein Recognition Research
计算机科学, 2022, 49(6A): 1-11. https://doi.org/10.11896/jsjkx.210400056
[7] 王君锋, 刘凡, 杨赛, 吕坦悦, 陈峙宇, 许峰.
基于多源迁移学习的大坝裂缝检测
Dam Crack Detection Based on Multi-source Transfer Learning
计算机科学, 2022, 49(6A): 319-324. https://doi.org/10.11896/jsjkx.210500124
[8] 彭云聪, 秦小林, 张力戈, 顾勇翔.
面向图像分类的小样本学习算法综述
Survey on Few-shot Learning Algorithms for Image Classification
计算机科学, 2022, 49(5): 1-9. https://doi.org/10.11896/jsjkx.210500128
[9] 高元浩, 罗晓清, 张战成.
基于特征分离的红外与可见光图像融合算法
Infrared and Visible Image Fusion Based on Feature Separation
计算机科学, 2022, 49(5): 58-63. https://doi.org/10.11896/jsjkx.210200148
[10] 谭珍琼, 姜文君, 任演纳, 张吉, 任德盛, 李晓鸿.
基于二分图的个性化学习任务分配
Personalized Learning Task Assignment Based on Bipartite Graph
计算机科学, 2022, 49(4): 269-281. https://doi.org/10.11896/jsjkx.210500125
[11] 左杰格, 柳晓鸣, 蔡兵.
基于图像分块与特征融合的户外图像天气识别
Outdoor Image Weather Recognition Based on Image Blocks and Feature Fusion
计算机科学, 2022, 49(3): 197-203. https://doi.org/10.11896/jsjkx.201200263
[12] 张舒萌, 余增, 李天瑞.
跨领域文本的可迁移情绪分析方法
Transferable Emotion Analysis Method for Cross-domain Text
计算机科学, 2022, 49(3): 218-224. https://doi.org/10.11896/jsjkx.210400034
[13] 李星燃, 张立言, 姚树婧.
结合特征融合和注意力机制的微表情识别方法
Micro-expression Recognition Method Combining Feature Fusion and Attention Mechanism
计算机科学, 2022, 49(2): 4-11. https://doi.org/10.11896/jsjkx.210900028
[14] 任首朋, 李劲, 王静茹, 岳昆.
基于集成回归决策树的lncRNA-疾病关联预测方法
Ensemble Regression Decision Trees-based lncRNA-disease Association Prediction
计算机科学, 2022, 49(2): 265-271. https://doi.org/10.11896/jsjkx.201100132
[15] 侯宏旭, 孙硕, 乌尼尔.
蒙汉神经机器翻译研究综述
Survey of Mongolian-Chinese Neural Machine Translation
计算机科学, 2022, 49(1): 31-40. https://doi.org/10.11896/jsjkx.210900006
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!