计算机科学 ›› 2020, Vol. 47 ›› Issue (12): 177-182.doi: 10.11896/jsjkx.191000141

• 计算机图形学与多媒体 • 上一篇    下一篇

基于深度多任务学习的社交图像标签和分组联合推荐

耿蕾蕾1, 崔超然1, 石成2, 申朕2, 尹义龙3, 冯仕红1   

  1. 1 山东财经大学计算机科学与技术学院 济南 250014
    2 山东大学计算机科学与技术学院 济南 250014
    3 山东大学软件学院 济南 250014
  • 收稿日期:2019-10-22 修回日期:2020-03-22 发布日期:2020-12-17
  • 通讯作者: 崔超然(crcui@sdufe.edu.cn)
  • 作者简介:leileigeng_njust@163.com
  • 基金资助:
    国家自然科学基金(617012816187609861573219);国家重点研发计划课题(2018YFC08301002018YFC0830102);山东省高等学校科技计划项目(KJ2018BAN047);山东省高等学校"青创科技计划"立项支持(2019KJN045);山东省高等学校优势学科人才团队培育计划

Social Image Tag and Group Joint Recommendation Based on Deep Multi-task Learning

GENG Lei-lei1, CUI Chao-ran1, SHI Cheng2, SHEN Zhen2, YIN Yi-long3, FENG Shi-hong1   

  1. 1 School of Computer Science and Technology Shandong University of Finance and EconomicsJinan 250014,China
    2 School of Computer Science and Technology Shandong University Jinan 250014,China
    3 School of Software Shandong UniversityJinan 250014,China
  • Received:2019-10-22 Revised:2020-03-22 Published:2020-12-17
  • About author:GENG Lei-lei,born in 1984Ph.Dassociate professoris a member of China Computer Federation.Her main research interests include machine learning and computer vision.
  • Supported by:
    National Natural Science Fund of China (61701281,61876098,61573219),National Key R&D Program of China (2018YFC0830100,2018YFC0830102),Shandong Province Higher Educational Science and Technology Program (KJ2018BAN047),Science and Technology Innovation Program for Distributed Young Talents of Shandong Province Higher Education Institutions (2019KJN045) and Fostering Project of Dominant Discipline and Talent Team of Shandong Province Higher Education Institutions.

摘要: 随着在线社交的多媒体共享网站的蓬勃发展社交图像推荐逐渐成为研究热点.人们通常对社交图像进行标签化、分组化使得图像数据更加易于管理.传统的图像标签或分组推荐方法往往只关注特定任务忽略了标签推荐和分组推荐任务之间的隐含关系.多任务学习则可以充分挖掘不同任务对图像的共享或相互关联的隐含表示融合多种任务抽取图像特征对于提高单一任务的准确性具有积极意义.因此文中提出了一种基于深度多任务学习的社交图像标签和分组联合推荐模型.该方法使用基于比较的偏序学习深度网络分别进行标签推荐和分组推荐有效缓解了单任务中的数据稀疏性问题.此外在处理社交图像视觉特征的卷积神经网络中首先使用多任务学习将来自不同任务的中间层特征进行连接然后通过卷积实现降维和特征的自动融合使得不同任务的图像特征得到共享同时降维后的融合特征能够满足下一层卷积神经网络的尺寸要求使得单一任务的整体结构得以保持.从大量Flickr图片共享网站上爬取的真实数据集上的实验结果表明与现有经典推荐算法相比所提算法获得的准确率和召回率均有较大提升证明了该方法的有效性和可行性.

关键词: 多任务学习, 分组推荐, 标签推荐, 偏序学习, 特征融合, 联合推荐

Abstract: With the rapid development of multimedia sharing websitesthe social image recommendation has become a hot research topic recently.It is much easier to manage social images by taging and grouping them.Traditional image recommendation methods tend to focus on a specific task just like tag or group recommendationwhich ignore the correlation between the two tasks.By fusing image features extracted from multiple tasksmulti-task learning method exploit image sharing or correlation representation among different tasks to improve the accuracy of the single task.Thereforethis paper proposes a novel social image tag and group joint recommendation model based on deep multi-task learning.In the signal taskthe tag and group recommendation are solved using comparison-based partial order deep learning method respectively for alleviating the data sparsity.Furthermorethe features extracted from their intermediate layer are saved for multi-task learning.In the convolutional neural network processing the visual features of social imagesthe features of two recommendations are connected.Then the dimension-reduction and automatic fusion of the features are realized by convolution.Hence image features extracted from different recommendations are shared.Moreoverthe size of processed features is suitable for the next layer of the convolutional neural network so that the network architecture of single recommendation task can be maintained.The experimental results on a real Flickr dataset show that compared with the traditional methodsthe accuracy and recall rate of the proposed algorithm are greatly improvedwhich proves the effectiveness and feasibility of the proposed method.

Key words: Multi-task learning, Group recommendation, Tag recommendation, Partial order learning, Feature fusion, Joint recommendation

中图分类号: 

  • TP391.3
[1] GAO Y D,HOU L Y,YANG D L.Automatic Image Annotation Method using Multi-Label Learning Convolutional Neural Network[J].Journal of Computer Applications,2017,37(1):228-232.
[2] CHEN F,HE Y,TANG L P.Emotional Recognition of Visual-perception Oriented Images and Its Application in the Recommendation System[J].Journal of the China Society for Scientific and Technical Information,2019,38(4):420-431.
[3] QI C.An Approach of Image Tag Recommendation based onSubspace Learning Model[J].Computer and Modernization,2016,3:68-73.
[4] ZHAO T L,LIU Z,HAN H J,et al.A Personalized Image Tag Recommendation Algorithm based on Bipartite Graph Model[J].Journal of Nanjing University (Natural Science),2018,54(6):1193-1205.
[5] NGUYEN H T H,WISTUBA M,GRABOCKA J,et al.Personalized Deep Learning for Tag Recommendation[C]//Pacific-Asia Conference on Knowledge Discovery and Data Mining.Springer,Cham,2017:186-197.
[6] XU Y B,SU S M,FAN L Q.User-Specific Method for Aesthetic Images Recommendation[J].Application Research of Compu-ters,2019,36(12):3853-3856.
[7] LI J C,YUAN C,SONG Y.Multi-label Image Annotation Based on Convolution Neural Network[C]//The 11th Harmonious Human-Machine Environment.2015:56-59.
[8] LIU L.A Sparse Image Recommendation Model using Content and User Preference Information[C]//IEEE International Conference on Web Intelligence.Piscataway,2016:232-239.
[9] ZHANG J,YANG Y,TIAN Q,et al.Personalized Social Image Recommendation Method based on User Image Tag Model[J].IEEE Transaction on Multimedia,2017,19(11):2439-2449.
[10] LONG M,CAO Z,WANG J,et al.Learning Multiple Taskswith Multilinear Relationship Networks[C]//Advances in Neural Information Processing Systems.2017:1595-1604.
[11] LU Y,KUMAR A,ZHAI S,et al.Fully-Adaptive Feature Sharing in Multi-Task Networks with Applications in Person Attribute Classification[C]//IEEE Conference on Computer Vision and Pattern Recognition.2017:1131-1140.
[12] MISRA I,SHRIVASTAVA A,GUPTA A,et al.Cross-stitchNetworks for Multi-Task Learning[C]//IEEE Conference on Computer Vision and Pattern Recognition.2016:3994-4003.
[13] LEI C,LIU D,LI W,et al.Comparative Deep Learning of Hybrid Representations for Image Recommendations[C]//IEEE Conference on Computer Vision and Pattern Recognition.2016:2545-2553.
[14] CHEN M M,XUE K J.Personalized Tag Recommendation Algorithm based on Improved Tensor Decomposition Model[J].Data Analysis and Knowledge Discovery,2017,3:38-45.
[15] ZHANG J,YANG Y,TIAN Q,et al.Personalized Social Image Recommendation Method based on User-Image-Tag Model[J].IEEE Transactions on Multimedia,2017,19(11):2439-2449.
[16] ZHANG W,LIU F,JIANG L,et al.Recommendation Based on Collaborative Filtering by Convolution Deep Learning Model Based on Label Weight Nearest Neighbor[C]//The 10th International Symposium on Computational Intelligence and Design (ISCID).IEEE,2018:504-507.
[1] 张帆, 贺文琪, 姬红兵, 李丹萍, 王磊. 基于块对角化表示的多视角字典对学习[J]. 计算机科学, 2021, 48(1): 233-240.
[2] 潘祖江, 刘宁, 张伟, 王建勇. 基于层次注意力机制的多任务疾病进展模型[J]. 计算机科学, 2020, 47(9): 185-189.
[3] 吴宏涛, 刘力源, 孟颖, 荣亚鹏, 李路凯. 动态多特征融合的道路遗洒物威胁度分析方法[J]. 计算机科学, 2020, 47(6A): 196-205.
[4] 黄勇韬, 严华. 结合注意力机制与特征融合的场景图生成模型[J]. 计算机科学, 2020, 47(6): 133-137.
[5] 朱威, 王图强, 陈悦峰, 何德峰. 基于多尺度残差网络的对象级边缘检测算法[J]. 计算机科学, 2020, 47(6): 144-150.
[6] 裴嘉震, 徐曾春, 胡平. 融合视点机制与姿态估计的行人再识别方法[J]. 计算机科学, 2020, 47(6): 164-169.
[7] 胡宇佳, 甘伟, 朱敏. 基于多特征融合的增强子-启动子相互作用预测综述[J]. 计算机科学, 2020, 47(5): 64-71.
[8] 周子钦, 严华. 基于多任务学习的有限样本多视角三维形状识别算法[J]. 计算机科学, 2020, 47(4): 125-130.
[9] 陈燕文,李坤,韩焱,王燕平. 基于MFCC和常数Q变换的乐器音符识别[J]. 计算机科学, 2020, 47(3): 149-155.
[10] 傅雪阳,孙琦,黄悦,丁兴号. 基于深度邻近连接网络的单幅图像去雨方法[J]. 计算机科学, 2020, 47(2): 106-111.
[11] 周鹏程,龚声蓉,钟珊,包宗铭,戴兴华. 基于深度特征融合的图像语义分割[J]. 计算机科学, 2020, 47(2): 126-134.
[12] 段建勇, 游世薪, 张梅, 王昊. 基于多特征融合的关键词抽取[J]. 计算机科学, 2020, 47(11A): 73-77.
[13] 陈训敏, 叶书函, 詹瑞. 基于多任务学习及由粗到精的卷积神经网络人群计数模型[J]. 计算机科学, 2020, 47(11A): 183-187.
[14] 马康, 娄静涛, 苏致远, 李永乐, 朱愿. 结合特征融合和尺度自适应的核相关滤波器目标跟踪算法研究[J]. 计算机科学, 2020, 47(11A): 224-230.
[15] 李浩翔, 李浩君. MACTEN:新型大规模布料纹理分类框架[J]. 计算机科学, 2020, 47(11A): 258-265.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 雷丽晖,王静. 可能性测度下的LTL模型检测并行化研究[J]. 计算机科学, 2018, 45(4): 71 -75 .
[2] 孙启,金燕,何琨,徐凌轩. 用于求解混合车辆路径问题的混合进化算法[J]. 计算机科学, 2018, 45(4): 76 -82 .
[3] 周燕萍,业巧林. 基于L1-范数距离的最小二乘对支持向量机[J]. 计算机科学, 2018, 45(4): 100 -105 .
[4] 崔琼,李建华,王宏,南明莉. 基于节点修复的网络化指挥信息系统弹性分析模型[J]. 计算机科学, 2018, 45(4): 117 -121 .
[5] 贾伟,华庆一,张敏军,陈锐,姬翔,王博. 基于改进粒子群优化的移动界面模式聚类算法[J]. 计算机科学, 2018, 45(4): 220 -226 .
[6] 侯彦娥,孔云峰,党兰学. 求解多车型校车路径问题的混合集合划分的GRASP算法[J]. 计算机科学, 2018, 45(4): 240 -246 .
[7] 李昊阳,符云清. 基于标签聚类与项目主题的协同过滤推荐算法[J]. 计算机科学, 2018, 45(4): 247 -251 .
[8] 梁俊斌,周翔,王田,李陶深. 移动低占空比无线传感网中数据收集的研究进展[J]. 计算机科学, 2018, 45(4): 19 -24 .
[9] 黄冬梅, 杜艳玲, 贺琪, 随宏运, 李瑶. 基于多属性最优化的海洋监测数据副本布局策略[J]. 计算机科学, 2018, 45(6): 72 -75 .
[10] 吴建霞, 杨永立. 一种降低FBMC-OQAM系统PAPR的算法[J]. 计算机科学, 2018, 45(6): 89 -95 .