计算机科学 ›› 2020, Vol. 47 ›› Issue (10): 114-120.doi: 10.11896/jsjkx.190900038

• 数据库&大数据&数据科学 • 上一篇    下一篇

基于增强注意力机制的神经协同过滤

康雁, 卜荣景, 李浩, 杨兵, 张亚钏, 陈铁   

  1. 云南大学软件学院 昆明650091
  • 收稿日期:2019-09-04 修回日期:2020-01-05 出版日期:2020-10-15 发布日期:2020-10-16
  • 通讯作者: 康雁(kangyan@ynu.edu.cn)
  • 基金资助:
    国家自然科学基金(61762092,61762089);云南省软件工程重点实验室开放基金项目(2017SE204)

Neural Collaborative Filtering Based on Enhanced-attention Mechanism

KANG Yan, BU Rong-jing, LI Hao, YANG Bing, ZHANG Ya-chuan, CHEN Tie   

  1. School of Software,Yunnan University,Kunming 650091,China
  • Received:2019-09-04 Revised:2020-01-05 Online:2020-10-15 Published:2020-10-16
  • About author:KANG Yan,born in 1972,Ph.D,postgraduate supervisor,is a member of China Computer Federation.Her main research interests include machine learning and software engineering.
  • Supported by:
    National Natural Science Foundation of China (61762092,61762089) and Yunnan Provincial Key Laboratory of Software Engineering Open Fund Project (2017SE204)

摘要: 推荐系统是解决信息过载问题的核心。现有的推荐框架研究面临着显式反馈数据稀疏和数据预处理难等问题,特别是对新用户和新项目进行推荐的性能有待进一步提高。随着深度学习的推进,基于深度学习的推荐成为了当前的研究热点,大量的实验证明了深度学习运用于推荐系统的有效性。文中在NCF的基础上提出了EANCF(Neural Collaborative Filtering based on Enhanced-Attention Mechanism),从隐式反馈数据的角度研究了推荐框架,利用最大池化、局部推理以及组合多种不同数据融合方式来考虑数据特征提取;同时,引入注意力机制来为网络合理地分配权重值,减少信息的损失,提升推荐的性能。最后,基于两个大型真实数据集Movielens-1m和Pinterest-20对EANCF、NCF和部分经典算法做了对比实验,并且详细地给出了EANCF框架的训练过程。实验结果表明,EANCF框架确实具有较好的推荐性能,相比于NCF框架在HR@10和NDCG@10上均有显著提升,HR@10最高提升了3.53%,NDCG@10最高提升了2.47%。

关键词: 深度学习, 协同过滤, 隐式反馈, 注意力机制

Abstract: The recommendation system is the core to solve the problem of information overload.The existing research on recommendation framework faces many problems,such as sparse explicit feedback data and difficulty to preprocess data,especially the recommendation performances for new users and new projects need to be further improved.With the advancement of deep lear-ning,recommendation based on deep learning has become a current research hotspot.A large number of experiments have proved the effectiveness of deep learning applied to recommendation system.This paper presents EANCF (Neural Collaborative Filtering based on Enhanced-attention Mechanism) on the basis of NCF.It studies the recommendation framework from the perspective of implicit feedback data,and considers the data feature extraction by means of max-pooling,local inference modeling and combining many different ways of data fusion.Meanwhile,attention mechanism is introduced to reasonably allocate weight value for the network,reduce the loss of information and improve the performance of recommendation.Finally,based on two large real data sets,Movielens-1m and Pinterest-20,comparative experiments are carried out between EANCF and NCF,as well as some classical algorithms,and the training process of EANCF framework is given in detail.The experimental results show that the proposed EANCF framework does have good recommendation performance.Compared with the NCF,both HR@10 and NDCG@10 are significantly improved,with the highest increase of 3.53% for HR@10 and 2.47% for NDCG@10.

Key words: Attention mechanism, Collaborative filtering, Deep learning, Implicit feedback

中图分类号: 

  • TP391
[1]HE X N.Neural collaborative filtering[C]//Proceedings of the 26th International Conference on World Wide Web.2017:173-182.
[2]HE X,DU X,WANG X,et al.Outer Product-based Neural Collaborative Filtering[C]//Twenty-Seventh International Joint Conference on Artificial Intelligence.2018.
[3]BAI T,WEN J R,ZHANG J,et al.A neural collaborative filtering model with interaction-based neighborhood[C]//Proceedings of the 2017 ACM on Conference on Information and Knowledge Management.ACM,2017:1979-1982.
[4]WANG X,HE X,WANG M,et al.Neural graph collaborativefiltering[C]//Proceedings of the 42nd international ACM SIGIR conference on Research and development in Information Retrieval.2019:165-174.
[5]JIANG Y.Fusion Information Recommendation Model Based on Graph Convolution and Neural Collaborative Filtering [D].Changchun:Jilin University,2018.
[6]XIE E N,HE L M,WANG X H.Deep Collaborative Filtering Model Based on Attention Mechanism [J].Journal of Metrology University of China,2019,30(2):219-225,242.
[7]PORTEOUS I,ASUNCION A,WELLING M.Bayesian matrix factorization with side information and dirichlet process mixtures[C]//Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence.2010.
[8]CHEN Q,ZHU X,LING Z,et al.Enhanced LSTM for Natural Language Inference[J].arxiv:1609.06038,2016.
[9]OARD D W,KIM J.Implicit feedback for recommender systems[C]//Proceedings of the AAAI Workshop on Recommender Systems.1998.
[10]YAO W L,HE J,HUANG G Y,et al.A Graph-based model for context-aware recommendation using implicit feedback data[J].World Wide Web,2015,18(5):1351-1371.
[11]HE M,MORIMOTO Y.Capturing Temporal Dynamics of Implicit Feedbacks for Collaborative Filtering by Using Deep Recurrent Neural Networks[J].Bulletin of Networking,Computing,Systems,and Software,2018,7(1):33-37.
[12]GOLDBERG D,NICHOLS D,OKI B M,et al.Using collaborative filtering to weave an information tapestry[J].Communications of the ACM,1992,35(12):61-70.
[13]RESNICK P,IACOVOU N,SUCHAK M,et al.GroupLens:an open architecture for collaborative filtering of netnews[C]//Proceedings of the 1994 ACM Conference on Computer Supported Cooperative Work.ACM,1994:175-186.
[14]ZHAO X Y,XIA L,ZHANG L,et al.Deep reinforcement lear-ning for page-wise recommendations[C]//Proceedings of the 12th ACM Conference on Recommender Systems.2018:95-103.
[15]PASRICHA R,MCAULEY J.Translation-based factorizationmachines for sequential recommendation[C]//Proceedings of the 12th ACM Conference on Recommender Systems.2018:63-71.
[16]SUN Z,YANG J,ZHANG J,et al.Recurrent knowledge graph embedding for effective recommendation[C]//Proceedings of the 12th ACM Conference on Recommender Systems.2018:297-305.
[17]DUAN D K,FU X F.Research on User Cold Start Problem in Hybrid Collaborative Filtering Algorithm [J].CEA,2017,53(21):151-156.
[18]ZHANG X C.Utilizing Tri-training Algorithm to Solve ColdStart Problem in Recommender System[J].Computer Science,2016,43(12):108-114.
[19]DENG C B,YU H Q,FAN G S.Integrating Dynamic Collaborative Filtering and Deep Learning for Recommendation[J].Computer Science,2019,46(8):28-34.
[20]ZHENG L.A survey and critique of deep learning on recommender systems[J].Chicago:University of Illinois,2016:31.
[21]LONG C,ZHANG H,XIAO J,et al.SCA-CNN:Spatial andChannel-Wise Attention in Convolutional Networks for Image Captioning[C]//IEEE Conference on Computer Vision & Pattern Recognition.2017:5659-5667.
[22]BAHDANAU D,CHO K,BENGIO Y.Neural Machine Translation by Jointly Learning to Align and Translate[J].arxiv:1409.0473,2014.
[23]LI J,REN P,CHEN Z,et al.Neural attentive session-based reco-mmendation[C]//Proceedings of the 2017 ACM on Conference on Information and Knowledge Management.2017:1419-1428.
[24]GUO X,ZHU J H.Deep Neural Network RecommendationModel Based on User Vectorization Representation and Attention Mechanism[J].Computer Science,2019,46(8):111-115.
[25]DACREMA M F,CREMONESI P,JANNACH D.Are we really making much progress? A worrying analysis of recent neural recommendation approaches[C]//Proceedings of the 13th ACM Conference on Recommender Systems.ACM,2019:101-109.
[1] 饶志双, 贾真, 张凡, 李天瑞.
基于Key-Value关联记忆网络的知识图谱问答方法
Key-Value Relational Memory Networks for Question Answering over Knowledge Graph
计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[2] 汤凌韬, 王迪, 张鲁飞, 刘盛云.
基于安全多方计算和差分隐私的联邦学习方案
Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy
计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[3] 程章桃, 钟婷, 张晟铭, 周帆.
基于图学习的推荐系统研究综述
Survey of Recommender Systems Based on Graph Learning
计算机科学, 2022, 49(9): 1-13. https://doi.org/10.11896/jsjkx.210900072
[4] 王冠宇, 钟婷, 冯宇, 周帆.
基于矢量量化编码的协同过滤推荐方法
Collaborative Filtering Recommendation Method Based on Vector Quantization Coding
计算机科学, 2022, 49(9): 48-54. https://doi.org/10.11896/jsjkx.210700109
[5] 周芳泉, 成卫青.
基于全局增强图神经网络的序列推荐
Sequence Recommendation Based on Global Enhanced Graph Neural Network
计算机科学, 2022, 49(9): 55-63. https://doi.org/10.11896/jsjkx.210700085
[6] 戴禹, 许林峰.
基于文本行匹配的跨图文本阅读方法
Cross-image Text Reading Method Based on Text Line Matching
计算机科学, 2022, 49(9): 139-145. https://doi.org/10.11896/jsjkx.220600032
[7] 周乐员, 张剑华, 袁甜甜, 陈胜勇.
多层注意力机制融合的序列到序列中国连续手语识别和翻译
Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion
计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026
[8] 徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺.
时序知识图谱表示学习
Temporal Knowledge Graph Representation Learning
计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[9] 熊丽琴, 曹雷, 赖俊, 陈希亮.
基于值分解的多智能体深度强化学习综述
Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization
计算机科学, 2022, 49(9): 172-182. https://doi.org/10.11896/jsjkx.210800112
[10] 王剑, 彭雨琦, 赵宇斐, 杨健.
基于深度学习的社交网络舆情信息抽取方法综述
Survey of Social Network Public Opinion Information Extraction Based on Deep Learning
计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[11] 郝志荣, 陈龙, 黄嘉成.
面向文本分类的类别区分式通用对抗攻击方法
Class Discriminative Universal Adversarial Attack for Text Classification
计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[12] 姜梦函, 李邵梅, 郑洪浩, 张建朋.
基于改进位置编码的谣言检测模型
Rumor Detection Model Based on Improved Position Embedding
计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[13] 朱承璋, 黄嘉儿, 肖亚龙, 王晗, 邹北骥.
基于注意力机制的医学影像深度哈希检索算法
Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism
计算机科学, 2022, 49(8): 113-119. https://doi.org/10.11896/jsjkx.210700153
[14] 孙奇, 吉根林, 张杰.
基于非局部注意力生成对抗网络的视频异常事件检测方法
Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection
计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[15] 闫佳丹, 贾彩燕.
基于双图神经网络信息融合的文本分类方法
Text Classification Method Based on Information Fusion of Dual-graph Neural Network
计算机科学, 2022, 49(8): 230-236. https://doi.org/10.11896/jsjkx.210600042
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!