计算机科学 ›› 2022, Vol. 49 ›› Issue (8): 70-77.doi: 10.11896/jsjkx.210600011

• 数据库&大数据&数据科学* 上一篇    下一篇

基于自注意力机制和迁移学习的跨领域推荐算法

方义秋1, 张震坤1, 葛君伟2   

  1. 1 重庆邮电大学计算机科学与技术学院 重庆 400065
    2 重庆邮电大学软件工程学院 重庆 400065
  • 收稿日期:2021-06-01 修回日期:2021-09-03 发布日期:2022-08-02
  • 通讯作者: 张震坤(zhangzhenkun@cqupt.email.cn)
  • 作者简介:(fangyq@cqupt.edu.cn)
  • 基金资助:
    国家自然科学基金(62072066)

Cross-domain Recommendation Algorithm Based on Self-attention Mechanism and Transfer Learning

FANG Yi-qiu1, ZHANG Zhen-kun1, GE Jun-wei2   

  1. 1 School of Computer Science and Technology,Chongqing University of Posts and Telecommunications,Chongqing 400065,China
    2 School of Software Engineering,Chongqing University of Posts and Telecommunications,Chongqing 400065,China
  • Received:2021-06-01 Revised:2021-09-03 Published:2022-08-02
  • About author:FANG Yi-qiu,born in 1963,master,associate professor,graduate supervisor.Her main research interests include cloud computing and big data.
    ZHANG Zhen-kun,born in 1997,master.His main research interests include big data and so on.
  • Supported by:
    National Natural Science Foundation of China(62072066).

摘要: 传统的单领域推荐算法受限于用户和项目的稀疏关系,存在用户/项目冷启动的问题,并且,其仅以用户对项目评分进行建模,忽略了评论文本中所蕴含的信息。基于评论文本的跨领域推荐算法在辅助领域提取用户/项目的评论信息来缓解目标领域的数据稀疏问题,以提高推荐的准确率。文中提出了结合自注意力机制和迁移学习的跨领域推荐算法SAMTL(Self-Attention Mechanism and Transfer Learning)。与现有算法不同,SAMTL充分融合了目标领域和辅助领域的知识。首先,引入自注意力机制建模用户的喜好信息;其次,通过交叉映射跨域传输网络实现借助一个领域的信息来提高另一个领域的推荐准确率;最后,在知识融合模块和评分预测模块整合两个域的信息,进行评分预测。在Amazon数据集上的实验表明,与现有的跨领域推荐模型相比,SAMTL的MAEMSE值更高,在3种不同的跨领域数据集上的MAE值分别提高了8.4%,13.2%和19.4%,MSE值分别提高了6.3%,7.8%和5.6%。通过多项实验验证了自注意力机制和迁移学习的有效性,以及它们在缓解数据稀疏和用户冷启动问题方面的优势。

关键词: 跨领域, 评论文本, 迁移学习, 推荐系统, 自注意力

Abstract: Traditional single-domain recommendation algorithm is limited by the sparse relationship between users and items,and there is a problem of user/item cold start,and only models the item ratings by users,ignoring the information contained in the review text.The cross-domain recommendation algorithm based on review text extracts user/item review information in the auxiliary domain to alleviate data sparseness in the target domain and improve the accuracy of recommendation.This paper proposes a cross-domain recommendation algorithm SAMTL that combines self-attention mechanism and transfer learning.Different from existing algorithms,SAMTL fully integrates the knowledge of the target domain and auxiliary domains.Firstly,the self-attention mechanism is introduced to model user’s preference information.Then,by the cross-mapping cross-domain transmission network,the recommendation accuracy of another domain is improved with the help of information in one domain.Finally,the information of the two domains is integrated in the knowledge fusion and scoring prediction module to perform scoring prediction.Experiments on Amazon data set show that,compared with the existing cross-domain recommendation model,SAMTL has higher MAE and MSE values,and MAE increases by 8.4%,13.2% and 19.4% on three different cross-domain data sets,MSE increases by 6.3%,7.8% and 5.6% respectively.A number of experiments verify the effectiveness of self-attention mechanism and transfer learning,as well as the advantages in alleviating data sparsity and user cold start problems.

Key words: Comment text, Cross-domain, Recommendation system, Self-attention, Transfer learning

中图分类号: 

  • TP391
[1]MOONEY R J,ROY L.Content-Based Book RecommendingUsing Learning for Text Categorization[J].arXiv:cs/9902011,1999.
[2]BALABANOVIC M,SHOHAM Y.Fab:content-based,collaborative recommendation[J].Communications of the ACM,1997,40(3):66-72.
[3]SALAKHUTDINOV R,MNIH A,HINTON G.Restricted Boltzmann machines for collaborative filtering[C]//Proceedings of the 24th International Conference on Machine Learning.2007:791-798.
[4]ZHU F,WANG Y,CHEN C,et al.A deep framework for cross-domain and cross-system recommendations[C]//IJCAI International Joint Conference on Artificial Intelligence.2018:3711-3717.
[5]KHAN M M,IBRAHIM R,GHANI I.Cross domain recommender systems:a systematic literature review[J].ACM Computing Surveys(CSUR),2017,50(3):1-34.
[6]PAN W,XIANG E,LIU N,et al.Transfer learning in collaborative filtering for sparsity reduction[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2010:230-235.
[7]MAN T,SHEN H,JIN X,et al.Cross-Domain Recommendation:An Embedding and Mapping Approach[C]//IJCAI.2017:2464-2470.
[8]WANG X,PENG Z,WANG S,et al.Cross-domain Recommendation for Cold-start Users via Neighborhood based feature mapping[C]//International Conference on Database Systems for Advanced Applications.Cham:Springer,2018:158-165.
[9]FU W,PENG Z,WANG S,et al.Deeply Fusing Reviews andContents for Cold Start Users in Cross-Domain Recommendation Systems[J].Proceedings of the AAAI Conference on Artificial Intelligence,2019,33:94-101.
[10]LI C,QUAN C,PENG L,et al.A capsule network for recommendation and explaining what you like and dislike[C]//Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval.2019:275-284.
[11]LONI B,SHI Y,LARSON M,et al.Cross-domain collaborative filtering with factorization machines[C]//European Conference on Information Retrieval.Cham:Springer,2014:656-661.
[12]LI B,QIANG Y,XUE X.Can movies and books collaborate?cross-domain collaborative filtering for sparsity reduction[C]//Twenty-First International Joint Conference on Artificial Intelligence.2009:2052-2057.
[13]LI B,YANG Q,XUE X.Transfer learning for collaborative filtering via a rating-matrix generative model[C]//Proceedings of the 26th Annual International Conference on Machine Learning.2009:617-624.
[14]PAN W,XIANG E,YANG Q.Transfer learning in collaborative filtering with uncertain ratings[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2012:662-668.
[15]KARATZOGLOU A,AMATRIAIN X,BALTRUNAS L,et al.Multiverse recommendation:N-dimensional tensor factorization for context-aware collaborative filtering[C]//Proceedings of the Fourth ACM Conference on Recommender Systems.2010:79-86.
[16]WANG J,LI S J,YANG S,et al.A new transfer learning model for cross-domain recommendation[J].Chinese Journal of Computers,2017,40(33):1-15.
[17]SHI Y,LARSON M,HANJALIC A.Tags as bridges between domains:Improving recommendation with tag-induced cross-domain collaborative filtering[C]//International Conference on User Modeling,Adaptation,and Personalization.Berlin:Sprin-ger,2011:305-316.
[18]SINGH A P,GORDON G J.Relational learning via collectivematrix factorization[C]//Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.2008:650-658.
[19]ELKAHKY A M,SONG Y,HE X.A multi-view deep learning approach for cross domain user modeling in recommendation systems[C]//Proceedings of the 24th International Conference on World Wide Web.2015:278-288.
[20]HU L,CAO J,XU G,et al.Personalized recommendation viacross-domain triadic factorization[C]//Proceedings of the 22nd International Conference on World Wide Web.2013:595-606.
[21]WU J,XU J,DING T.Fine-grained image classification algo-rithm based on ensemble methods of transfer learning [J].Journal of Chongqing University of Posts and Telecommunications(Natural Science Edition),2020,32(3):452-458.
[22]PENNINGTON J,SOCHER R,MANNING C D.Glove:Global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Proces-sing(EMNLP).2014:1532-1543.
[23]HU G,ZHANG Y,YANG Q.Conet:Collaborative cross networks for cross-domain recommendation[C]//Proceedings of the 27th ACM International Conference on Information and Knowledge Management.2018:667-676.
[24]NAIR V,HINTON G E.Rectified linear units improve restric-ted boltzmann machines[C]//ICML.2010:807-814.
[25]SEO S,HUANG J,YANG H,et al.Interpretable convolutional neural networks with dual local and global attention for review rating prediction[C]//Proceedings of the eleventh ACM Confe-rence on Recommender Systems.2017:297-305.
[26]CHIN J Y,ZHAN K,JOTY S,et al.ANR:Aspect-based neural recommender[C]//Proceedings of the 27th ACM International Conference on Information and Knowledge Management.2018:147-156.
[1] 程章桃, 钟婷, 张晟铭, 周帆.
基于图学习的推荐系统研究综述
Survey of Recommender Systems Based on Graph Learning
计算机科学, 2022, 49(9): 1-13. https://doi.org/10.11896/jsjkx.210900072
[2] 张佳, 董守斌.
基于评论方面级用户偏好迁移的跨领域推荐算法
Cross-domain Recommendation Based on Review Aspect-level User Preference Transfer
计算机科学, 2022, 49(9): 41-47. https://doi.org/10.11896/jsjkx.220200131
[3] 王冠宇, 钟婷, 冯宇, 周帆.
基于矢量量化编码的协同过滤推荐方法
Collaborative Filtering Recommendation Method Based on Vector Quantization Coding
计算机科学, 2022, 49(9): 48-54. https://doi.org/10.11896/jsjkx.210700109
[4] 吴子仪, 李邵梅, 姜梦函, 张建朋.
基于自注意力模型的本体对齐方法
Ontology Alignment Method Based on Self-attention
计算机科学, 2022, 49(9): 215-220. https://doi.org/10.11896/jsjkx.210700190
[5] 秦琪琦, 张月琴, 王润泽, 张泽华.
基于知识图谱的层次粒化推荐方法
Hierarchical Granulation Recommendation Method Based on Knowledge Graph
计算机科学, 2022, 49(8): 64-69. https://doi.org/10.11896/jsjkx.210600111
[6] 陈坤峰, 潘志松, 王家宝, 施蕾, 张锦.
基于双目叠加仿生的微换衣行人再识别
Moderate Clothes-Changing Person Re-identification Based on Bionics of Binocular Summation
计算机科学, 2022, 49(8): 165-171. https://doi.org/10.11896/jsjkx.210600140
[7] 金方焱, 王秀利.
融合RACNN和BiLSTM的金融领域事件隐式因果关系抽取
Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM
计算机科学, 2022, 49(7): 179-186. https://doi.org/10.11896/jsjkx.210500190
[8] 帅剑波, 王金策, 黄飞虎, 彭舰.
基于神经架构搜索的点击率预测模型
Click-Through Rate Prediction Model Based on Neural Architecture Search
计算机科学, 2022, 49(7): 10-17. https://doi.org/10.11896/jsjkx.210600009
[9] 齐秀秀, 王佳昊, 李文雄, 周帆.
基于概率元学习的矩阵补全预测融合算法
Fusion Algorithm for Matrix Completion Prediction Based on Probabilistic Meta-learning
计算机科学, 2022, 49(7): 18-24. https://doi.org/10.11896/jsjkx.210600126
[10] 蔡晓娟, 谭文安.
一种改进的融合相似度和信任度的协同过滤算法
Improved Collaborative Filtering Algorithm Combining Similarity and Trust
计算机科学, 2022, 49(6A): 238-241. https://doi.org/10.11896/jsjkx.210400088
[11] 何亦琛, 毛宜军, 谢贤芬, 古万荣.
基于点割集图分割的矩阵变换与分解的推荐算法
Matrix Transformation and Factorization Based on Graph Partitioning by Vertex Separator for Recommendation
计算机科学, 2022, 49(6A): 272-279. https://doi.org/10.11896/jsjkx.210600159
[12] 王君锋, 刘凡, 杨赛, 吕坦悦, 陈峙宇, 许峰.
基于多源迁移学习的大坝裂缝检测
Dam Crack Detection Based on Multi-source Transfer Learning
计算机科学, 2022, 49(6A): 319-324. https://doi.org/10.11896/jsjkx.210500124
[13] 张嘉淏, 刘峰, 齐佳音.
一种基于Bottleneck Transformer的轻量级微表情识别架构
Lightweight Micro-expression Recognition Architecture Based on Bottleneck Transformer
计算机科学, 2022, 49(6A): 370-377. https://doi.org/10.11896/jsjkx.210500023
[14] 洪志理, 赖俊, 曹雷, 陈希亮, 徐志雄.
基于遗憾探索的竞争网络强化学习智能推荐方法研究
Study on Intelligent Recommendation Method of Dueling Network Reinforcement Learning Based on Regret Exploration
计算机科学, 2022, 49(6): 149-157. https://doi.org/10.11896/jsjkx.210600226
[15] 郭亮, 杨兴耀, 于炯, 韩晨, 黄仲浩.
基于注意力机制和门控网络相结合的混合推荐系统
Hybrid Recommender System Based on Attention Mechanisms and Gating Network
计算机科学, 2022, 49(6): 158-164. https://doi.org/10.11896/jsjkx.210500013
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!