Computer Science ›› 2020, Vol. 47 ›› Issue (12): 177-182.doi: 10.11896/jsjkx.191000141

Previous Articles     Next Articles

Social Image Tag and Group Joint Recommendation Based on Deep Multi-task Learning

GENG Lei-lei1, CUI Chao-ran1, SHI Cheng2, SHEN Zhen2, YIN Yi-long3, FENG Shi-hong1   

  1. 1 School of Computer Science and Technology Shandong University of Finance and EconomicsJinan 250014,China
    2 School of Computer Science and Technology Shandong University Jinan 250014,China
    3 School of Software Shandong UniversityJinan 250014,China
  • Received:2019-10-22 Revised:2020-03-22 Published:2020-12-17
  • About author:GENG Lei-lei,born in 1984Ph.Dassociate professoris a member of China Computer Federation.Her main research interests include machine learning and computer vision.
  • Supported by:
    National Natural Science Fund of China (61701281,61876098,61573219),National Key R&D Program of China (2018YFC0830100,2018YFC0830102),Shandong Province Higher Educational Science and Technology Program (KJ2018BAN047),Science and Technology Innovation Program for Distributed Young Talents of Shandong Province Higher Education Institutions (2019KJN045) and Fostering Project of Dominant Discipline and Talent Team of Shandong Province Higher Education Institutions.

Abstract: With the rapid development of multimedia sharing websitesthe social image recommendation has become a hot research topic recently.It is much easier to manage social images by taging and grouping them.Traditional image recommendation methods tend to focus on a specific task just like tag or group recommendationwhich ignore the correlation between the two tasks.By fusing image features extracted from multiple tasksmulti-task learning method exploit image sharing or correlation representation among different tasks to improve the accuracy of the single task.Thereforethis paper proposes a novel social image tag and group joint recommendation model based on deep multi-task learning.In the signal taskthe tag and group recommendation are solved using comparison-based partial order deep learning method respectively for alleviating the data sparsity.Furthermorethe features extracted from their intermediate layer are saved for multi-task learning.In the convolutional neural network processing the visual features of social imagesthe features of two recommendations are connected.Then the dimension-reduction and automatic fusion of the features are realized by convolution.Hence image features extracted from different recommendations are shared.Moreoverthe size of processed features is suitable for the next layer of the convolutional neural network so that the network architecture of single recommendation task can be maintained.The experimental results on a real Flickr dataset show that compared with the traditional methodsthe accuracy and recall rate of the proposed algorithm are greatly improvedwhich proves the effectiveness and feasibility of the proposed method.

Key words: Feature fusion, Group recommendation, Joint recommendation, Multi-task learning, Partial order learning, Tag recommendation

CLC Number: 

  • TP391.3
[1] GAO Y D,HOU L Y,YANG D L.Automatic Image Annotation Method using Multi-Label Learning Convolutional Neural Network[J].Journal of Computer Applications,2017,37(1):228-232.
[2] CHEN F,HE Y,TANG L P.Emotional Recognition of Visual-perception Oriented Images and Its Application in the Recommendation System[J].Journal of the China Society for Scientific and Technical Information,2019,38(4):420-431.
[3] QI C.An Approach of Image Tag Recommendation based onSubspace Learning Model[J].Computer and Modernization,2016,3:68-73.
[4] ZHAO T L,LIU Z,HAN H J,et al.A Personalized Image Tag Recommendation Algorithm based on Bipartite Graph Model[J].Journal of Nanjing University (Natural Science),2018,54(6):1193-1205.
[5] NGUYEN H T H,WISTUBA M,GRABOCKA J,et al.Personalized Deep Learning for Tag Recommendation[C]//Pacific-Asia Conference on Knowledge Discovery and Data Mining.Springer,Cham,2017:186-197.
[6] XU Y B,SU S M,FAN L Q.User-Specific Method for Aesthetic Images Recommendation[J].Application Research of Compu-ters,2019,36(12):3853-3856.
[7] LI J C,YUAN C,SONG Y.Multi-label Image Annotation Based on Convolution Neural Network[C]//The 11th Harmonious Human-Machine Environment.2015:56-59.
[8] LIU L.A Sparse Image Recommendation Model using Content and User Preference Information[C]//IEEE International Conference on Web Intelligence.Piscataway,2016:232-239.
[9] ZHANG J,YANG Y,TIAN Q,et al.Personalized Social Image Recommendation Method based on User Image Tag Model[J].IEEE Transaction on Multimedia,2017,19(11):2439-2449.
[10] LONG M,CAO Z,WANG J,et al.Learning Multiple Taskswith Multilinear Relationship Networks[C]//Advances in Neural Information Processing Systems.2017:1595-1604.
[11] LU Y,KUMAR A,ZHAI S,et al.Fully-Adaptive Feature Sharing in Multi-Task Networks with Applications in Person Attribute Classification[C]//IEEE Conference on Computer Vision and Pattern Recognition.2017:1131-1140.
[12] MISRA I,SHRIVASTAVA A,GUPTA A,et al.Cross-stitchNetworks for Multi-Task Learning[C]//IEEE Conference on Computer Vision and Pattern Recognition.2016:3994-4003.
[13] LEI C,LIU D,LI W,et al.Comparative Deep Learning of Hybrid Representations for Image Recommendations[C]//IEEE Conference on Computer Vision and Pattern Recognition.2016:2545-2553.
[14] CHEN M M,XUE K J.Personalized Tag Recommendation Algorithm based on Improved Tensor Decomposition Model[J].Data Analysis and Knowledge Discovery,2017,3:38-45.
[15] ZHANG J,YANG Y,TIAN Q,et al.Personalized Social Image Recommendation Method based on User-Image-Tag Model[J].IEEE Transactions on Multimedia,2017,19(11):2439-2449.
[16] ZHANG W,LIU F,JIANG L,et al.Recommendation Based on Collaborative Filtering by Convolution Deep Learning Model Based on Label Weight Nearest Neighbor[C]//The 10th International Symposium on Computational Intelligence and Design (ISCID).IEEE,2018:504-507.
[1] ZHANG Ying-tao, ZHANG Jie, ZHANG Rui, ZHANG Wen-qiang. Photorealistic Style Transfer Guided by Global Information [J]. Computer Science, 2022, 49(7): 100-105.
[2] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[3] YU Shu-hao, ZHOU Hui, YE Chun-yang, WANG Tai-zheng. SDFA:Study on Ship Trajectory Clustering Method Based on Multi-feature Fusion [J]. Computer Science, 2022, 49(6A): 256-260.
[4] YANG Yue, FENG Tao, LIANG Hong, YANG Yang. Image Arbitrary Style Transfer via Criss-cross Attention [J]. Computer Science, 2022, 49(6A): 345-352.
[5] CHEN Yong-ping, ZHU Jian-qing, XIE Yi, WU Han-xiao, ZENG Huan-qiang. Real-time Helmet Detection Algorithm Based on Circumcircle Radius Difference Loss [J]. Computer Science, 2022, 49(6A): 424-428.
[6] SUN Jie-qi, LI Ya-feng, ZHANG Wen-bo, LIU Peng-hui. Dual-field Feature Fusion Deep Convolutional Neural Network Based on Discrete Wavelet Transformation [J]. Computer Science, 2022, 49(6A): 434-440.
[7] DU Li-jun, TANG Xi-lu, ZHOU Jiao, CHEN Yu-lan, CHENG Jian. Alzheimer's Disease Classification Method Based on Attention Mechanism and Multi-task Learning [J]. Computer Science, 2022, 49(6A): 60-65.
[8] LAN Ling-xiang, CHI Ming-min. Remote Sensing Change Detection Based on Feature Fusion and Attention Network [J]. Computer Science, 2022, 49(6): 193-198.
[9] FAN Xin-nan, ZHAO Zhong-xin, YAN Wei, YAN Xi-jun, SHI Peng-fei. Multi-scale Feature Fusion Image Dehazing Algorithm Combined with Attention Mechanism [J]. Computer Science, 2022, 49(5): 50-57.
[10] LI Fa-guang, YILIHAMU·Yaermaimaiti. Real-time Detection Model of Insulator Defect Based on Improved CenterNet [J]. Computer Science, 2022, 49(5): 84-91.
[11] DONG Qi-da, WANG Zhe, WU Song-yang. Feature Fusion Framework Combining Attention Mechanism and Geometric Information [J]. Computer Science, 2022, 49(5): 129-134.
[12] LI Peng-zu, LI Yao, Ibegbu Nnamdi JULIAN, SUN Chao, GUO Hao, CHEN Jun-jie. Construction and Classification of Brain Function Hypernetwork Based on Overlapping Group Lasso with Multi-feature Fusion [J]. Computer Science, 2022, 49(5): 206-211.
[13] ZHAO Kai, AN Wei-chao, ZHANG Xiao-yu, WANG Bin, ZHANG Shan, XIANG Jie. Intracerebral Hemorrhage Image Segmentation and Classification Based on Multi-taskLearning of Shared Shallow Parameters [J]. Computer Science, 2022, 49(4): 203-208.
[14] XU Hua-jie, QIN Yuan-zhuo, YANG Yang. Scene Recognition Method Based on Multi-level Feature Fusion and Attention Module [J]. Computer Science, 2022, 49(4): 209-214.
[15] GAO Xin-yue, TIAN Han-min. Droplet Segmentation Method Based on Improved U-Net Network [J]. Computer Science, 2022, 49(4): 227-232.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!