计算机科学 ›› 2020, Vol. 47 ›› Issue (12): 177-182.doi: 10.11896/jsjkx.191000141

• 计算机图形学与多媒体 • 上一篇    下一篇

基于深度多任务学习的社交图像标签和分组联合推荐

耿蕾蕾1, 崔超然1, 石成2, 申朕2, 尹义龙3, 冯仕红1   

  1. 1 山东财经大学计算机科学与技术学院 济南 250014
    2 山东大学计算机科学与技术学院 济南 250014
    3 山东大学软件学院 济南 250014
  • 收稿日期:2019-10-22 修回日期:2020-03-22 发布日期:2020-12-17
  • 通讯作者: 崔超然(crcui@sdufe.edu.cn)
  • 作者简介:leileigeng_njust@163.com
  • 基金资助:
    国家自然科学基金(617012816187609861573219);国家重点研发计划课题(2018YFC08301002018YFC0830102);山东省高等学校科技计划项目(KJ2018BAN047);山东省高等学校"青创科技计划"立项支持(2019KJN045);山东省高等学校优势学科人才团队培育计划

Social Image Tag and Group Joint Recommendation Based on Deep Multi-task Learning

GENG Lei-lei1, CUI Chao-ran1, SHI Cheng2, SHEN Zhen2, YIN Yi-long3, FENG Shi-hong1   

  1. 1 School of Computer Science and Technology Shandong University of Finance and EconomicsJinan 250014,China
    2 School of Computer Science and Technology Shandong University Jinan 250014,China
    3 School of Software Shandong UniversityJinan 250014,China
  • Received:2019-10-22 Revised:2020-03-22 Published:2020-12-17
  • About author:GENG Lei-lei,born in 1984Ph.Dassociate professoris a member of China Computer Federation.Her main research interests include machine learning and computer vision.
  • Supported by:
    National Natural Science Fund of China (61701281,61876098,61573219),National Key R&D Program of China (2018YFC0830100,2018YFC0830102),Shandong Province Higher Educational Science and Technology Program (KJ2018BAN047),Science and Technology Innovation Program for Distributed Young Talents of Shandong Province Higher Education Institutions (2019KJN045) and Fostering Project of Dominant Discipline and Talent Team of Shandong Province Higher Education Institutions.

摘要: 随着在线社交的多媒体共享网站的蓬勃发展社交图像推荐逐渐成为研究热点.人们通常对社交图像进行标签化、分组化使得图像数据更加易于管理.传统的图像标签或分组推荐方法往往只关注特定任务忽略了标签推荐和分组推荐任务之间的隐含关系.多任务学习则可以充分挖掘不同任务对图像的共享或相互关联的隐含表示融合多种任务抽取图像特征对于提高单一任务的准确性具有积极意义.因此文中提出了一种基于深度多任务学习的社交图像标签和分组联合推荐模型.该方法使用基于比较的偏序学习深度网络分别进行标签推荐和分组推荐有效缓解了单任务中的数据稀疏性问题.此外在处理社交图像视觉特征的卷积神经网络中首先使用多任务学习将来自不同任务的中间层特征进行连接然后通过卷积实现降维和特征的自动融合使得不同任务的图像特征得到共享同时降维后的融合特征能够满足下一层卷积神经网络的尺寸要求使得单一任务的整体结构得以保持.从大量Flickr图片共享网站上爬取的真实数据集上的实验结果表明与现有经典推荐算法相比所提算法获得的准确率和召回率均有较大提升证明了该方法的有效性和可行性.

关键词: 标签推荐, 多任务学习, 分组推荐, 联合推荐, 偏序学习, 特征融合

Abstract: With the rapid development of multimedia sharing websitesthe social image recommendation has become a hot research topic recently.It is much easier to manage social images by taging and grouping them.Traditional image recommendation methods tend to focus on a specific task just like tag or group recommendationwhich ignore the correlation between the two tasks.By fusing image features extracted from multiple tasksmulti-task learning method exploit image sharing or correlation representation among different tasks to improve the accuracy of the single task.Thereforethis paper proposes a novel social image tag and group joint recommendation model based on deep multi-task learning.In the signal taskthe tag and group recommendation are solved using comparison-based partial order deep learning method respectively for alleviating the data sparsity.Furthermorethe features extracted from their intermediate layer are saved for multi-task learning.In the convolutional neural network processing the visual features of social imagesthe features of two recommendations are connected.Then the dimension-reduction and automatic fusion of the features are realized by convolution.Hence image features extracted from different recommendations are shared.Moreoverthe size of processed features is suitable for the next layer of the convolutional neural network so that the network architecture of single recommendation task can be maintained.The experimental results on a real Flickr dataset show that compared with the traditional methodsthe accuracy and recall rate of the proposed algorithm are greatly improvedwhich proves the effectiveness and feasibility of the proposed method.

Key words: Feature fusion, Group recommendation, Joint recommendation, Multi-task learning, Partial order learning, Tag recommendation

中图分类号: 

  • TP391.3
[1] GAO Y D,HOU L Y,YANG D L.Automatic Image Annotation Method using Multi-Label Learning Convolutional Neural Network[J].Journal of Computer Applications,2017,37(1):228-232.
[2] CHEN F,HE Y,TANG L P.Emotional Recognition of Visual-perception Oriented Images and Its Application in the Recommendation System[J].Journal of the China Society for Scientific and Technical Information,2019,38(4):420-431.
[3] QI C.An Approach of Image Tag Recommendation based onSubspace Learning Model[J].Computer and Modernization,2016,3:68-73.
[4] ZHAO T L,LIU Z,HAN H J,et al.A Personalized Image Tag Recommendation Algorithm based on Bipartite Graph Model[J].Journal of Nanjing University (Natural Science),2018,54(6):1193-1205.
[5] NGUYEN H T H,WISTUBA M,GRABOCKA J,et al.Personalized Deep Learning for Tag Recommendation[C]//Pacific-Asia Conference on Knowledge Discovery and Data Mining.Springer,Cham,2017:186-197.
[6] XU Y B,SU S M,FAN L Q.User-Specific Method for Aesthetic Images Recommendation[J].Application Research of Compu-ters,2019,36(12):3853-3856.
[7] LI J C,YUAN C,SONG Y.Multi-label Image Annotation Based on Convolution Neural Network[C]//The 11th Harmonious Human-Machine Environment.2015:56-59.
[8] LIU L.A Sparse Image Recommendation Model using Content and User Preference Information[C]//IEEE International Conference on Web Intelligence.Piscataway,2016:232-239.
[9] ZHANG J,YANG Y,TIAN Q,et al.Personalized Social Image Recommendation Method based on User Image Tag Model[J].IEEE Transaction on Multimedia,2017,19(11):2439-2449.
[10] LONG M,CAO Z,WANG J,et al.Learning Multiple Taskswith Multilinear Relationship Networks[C]//Advances in Neural Information Processing Systems.2017:1595-1604.
[11] LU Y,KUMAR A,ZHAI S,et al.Fully-Adaptive Feature Sharing in Multi-Task Networks with Applications in Person Attribute Classification[C]//IEEE Conference on Computer Vision and Pattern Recognition.2017:1131-1140.
[12] MISRA I,SHRIVASTAVA A,GUPTA A,et al.Cross-stitchNetworks for Multi-Task Learning[C]//IEEE Conference on Computer Vision and Pattern Recognition.2016:3994-4003.
[13] LEI C,LIU D,LI W,et al.Comparative Deep Learning of Hybrid Representations for Image Recommendations[C]//IEEE Conference on Computer Vision and Pattern Recognition.2016:2545-2553.
[14] CHEN M M,XUE K J.Personalized Tag Recommendation Algorithm based on Improved Tensor Decomposition Model[J].Data Analysis and Knowledge Discovery,2017,3:38-45.
[15] ZHANG J,YANG Y,TIAN Q,et al.Personalized Social Image Recommendation Method based on User-Image-Tag Model[J].IEEE Transactions on Multimedia,2017,19(11):2439-2449.
[16] ZHANG W,LIU F,JIANG L,et al.Recommendation Based on Collaborative Filtering by Convolution Deep Learning Model Based on Label Weight Nearest Neighbor[C]//The 10th International Symposium on Computational Intelligence and Design (ISCID).IEEE,2018:504-507.
[1] 张颖涛, 张杰, 张睿, 张文强.
全局信息引导的真实图像风格迁移
Photorealistic Style Transfer Guided by Global Information
计算机科学, 2022, 49(7): 100-105. https://doi.org/10.11896/jsjkx.210600036
[2] 程成, 降爱莲.
基于多路径特征提取的实时语义分割方法
Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction
计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[3] 郁舒昊, 周辉, 叶春杨, 王太正.
SDFA:基于多特征融合的船舶轨迹聚类方法研究
SDFA:Study on Ship Trajectory Clustering Method Based on Multi-feature Fusion
计算机科学, 2022, 49(6A): 256-260. https://doi.org/10.11896/jsjkx.211100253
[4] 杨玥, 冯涛, 梁虹, 杨扬.
融合交叉注意力机制的图像任意风格迁移
Image Arbitrary Style Transfer via Criss-cross Attention
计算机科学, 2022, 49(6A): 345-352. https://doi.org/10.11896/jsjkx.210700236
[5] 杜丽君, 唐玺璐, 周娇, 陈玉兰, 程建.
基于注意力机制和多任务学习的阿尔茨海默症分类
Alzheimer's Disease Classification Method Based on Attention Mechanism and Multi-task Learning
计算机科学, 2022, 49(6A): 60-65. https://doi.org/10.11896/jsjkx.201200072
[6] 陈永平, 朱建清, 谢懿, 吴含笑, 曾焕强.
基于外接圆半径差损失的实时安全帽检测算法
Real-time Helmet Detection Algorithm Based on Circumcircle Radius Difference Loss
计算机科学, 2022, 49(6A): 424-428. https://doi.org/10.11896/jsjkx.220100252
[7] 孙洁琪, 李亚峰, 张文博, 刘鹏辉.
基于离散小波变换的双域特征融合深度卷积神经网络
Dual-field Feature Fusion Deep Convolutional Neural Network Based on Discrete Wavelet Transformation
计算机科学, 2022, 49(6A): 434-440. https://doi.org/10.11896/jsjkx.210900199
[8] 蓝凌翔, 池明旻.
基于特征注意力融合网络的遥感变化检测研究
Remote Sensing Change Detection Based on Feature Fusion and Attention Network
计算机科学, 2022, 49(6): 193-198. https://doi.org/10.11896/jsjkx.210500058
[9] 范新南, 赵忠鑫, 严炜, 严锡君, 史朋飞.
结合注意力机制的多尺度特征融合图像去雾算法
Multi-scale Feature Fusion Image Dehazing Algorithm Combined with Attention Mechanism
计算机科学, 2022, 49(5): 50-57. https://doi.org/10.11896/jsjkx.210400093
[10] 李发光, 伊力哈木·亚尔买买提.
基于改进CenterNet的航拍绝缘子缺陷实时检测模型
Real-time Detection Model of Insulator Defect Based on Improved CenterNet
计算机科学, 2022, 49(5): 84-91. https://doi.org/10.11896/jsjkx.210400142
[11] 董奇达, 王喆, 吴松洋.
结合注意力机制与几何信息的特征融合框架
Feature Fusion Framework Combining Attention Mechanism and Geometric Information
计算机科学, 2022, 49(5): 129-134. https://doi.org/10.11896/jsjkx.210300180
[12] 李鹏祖, 李瑶, Ibegbu Nnamdi JULIAN, 孙超, 郭浩, 陈俊杰.
基于多特征融合的重叠组套索脑功能超网络构建及分类
Construction and Classification of Brain Function Hypernetwork Based on Overlapping Group Lasso with Multi-feature Fusion
计算机科学, 2022, 49(5): 206-211. https://doi.org/10.11896/jsjkx.210300049
[13] 赵凯, 安卫超, 张晓宇, 王彬, 张杉, 相洁.
共享浅层参数多任务学习的脑出血图像分割与分类
Intracerebral Hemorrhage Image Segmentation and Classification Based on Multi-taskLearning of Shared Shallow Parameters
计算机科学, 2022, 49(4): 203-208. https://doi.org/10.11896/jsjkx.201000153
[14] 许华杰, 秦远卓, 杨洋.
基于多级特征融合与注意力模块的场景识别方法
Scene Recognition Method Based on Multi-level Feature Fusion and Attention Module
计算机科学, 2022, 49(4): 209-214. https://doi.org/10.11896/jsjkx.210100135
[15] 高心悦, 田汉民.
基于改进U-Net网络的液滴分割方法
Droplet Segmentation Method Based on Improved U-Net Network
计算机科学, 2022, 49(4): 227-232. https://doi.org/10.11896/jsjkx.210300193
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!