计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (5): 45-53.doi: 10.11896/jsjkx.230200049

• 数据库&大数据&数据科学 • 上一篇    下一篇

融入时间信息的预训练序列推荐方法

陈稳中1, 陈红梅1,2, 周丽华1,2, 方圆3   

  1. 1 云南大学信息学院 昆明 650500
    2 云南大学云南省智能系统与计算重点实验室 昆明 650500
    3 云南大学西南天文研究所 昆明 650500
  • 收稿日期:2023-02-08 修回日期:2023-05-24 出版日期:2024-05-15 发布日期:2024-05-08
  • 通讯作者: 陈红梅(hmchen@ynu.edu.cn)
  • 作者简介:(1040490734@qq.com)
  • 基金资助:
    国家自然科学基金(62266050,62276227);云南省中青年学术和技术带头人后备人才项目(202205AC160033);云南省基础研究计划重点项目(202201AS070015);云南省智能系统与计算重点实验室开放课题(ISC22Z02)

Time-aware Pre-training Method for Sequence Recommendation

CHEN Wenzhong1, CHEN Hongmei1,2, ZHOU Lihua1,2, FANG Yuan3   

  1. 1 School of Information Science and Engineering,Yunnan University,Kunming 650500,China
    2 Yunnan Key Laboratory of Intelligent Systems and Computing,Yunnan University,Kunming 650500,China
    3 South-Western Institute for Astronomy Research,Yunnan University,Kunming 650500,China
  • Received:2023-02-08 Revised:2023-05-24 Online:2024-05-15 Published:2024-05-08
  • About author:CHEN Wenzhong,born in 1994,postgraduate,is a member of CCF(No.J6048G).His main research interests include recommendation system and heterogeneous information network analysis.
    CHEN Hongmei,born in 1976,Ph.D,associate professor,is a member of CCF(No.49450M).Her main research interests include spatial data mining and location-based social network analysis.
  • Supported by:
    National Natural Science Foundation of China(62266050,62276227),Program for Young and Middle-aged Academic and Technical Reserve Leaders of Yunnan Province(202205AC160033),Yunnan Fundamental Research Projects(202201AS070015) and Open Project Program of Yunnan Key Laboratory of Intelligent Systems and Computing(ISC22Z02).

PDF (PC)

375

摘要： 序列推荐旨在根据用户与项目的历史交互序列,学习用户动态偏好,为用户推荐后续可能感兴趣的项目。基于预训练模型在适应下游任务方面具有优势,预训练机制在序列推荐中备受关注。现有序列推荐预训练方法忽略了现实中时间对用户交互行为的影响,为了更好地捕获用户与项目交互的时间语义,提出了融入时间信息的预训练序列推荐模型TPTS-Rec(Time-aware Pre-Training method for Sequence Recommendation)。首先,在嵌入层引入时间嵌入矩阵以获取用户交互项目与时间的关联信息。然后,在自注意力层采用同一时间点采样的方法以学习项目间的时间关联信息。最后,在微调阶段从时间维度扩增用户交互序列长度以缓解数据稀疏性问题。在真实数据集上的对比实验结果表明,与基线模型相比,所提模型TPTS-Rec的推荐效果有显著提升。

关键词: 序列推荐, 预训练, 自监督学习, 互信息最大化, 时间属性

Abstract: Sequence recommendation aims to learn users' dynamic preferences and recommend the next items which users may be interested in by analyzing historical interaction sequences between users and items.The pre-training model has attracted attentions from researchers in sequence recommendation due to its advantage of being adapted for downstream tasks.The existing pre-training methods for sequence recommendation ignore the impact of time on user interaction behaviors in real life.To better capture the time semantics of interactions between users and items,this paper proposes a novel model TPTS-Rec(time-aware pre-training method for sequence recommendation).First,the time embedding matrix is introduced in the embedding layer to obtain the correlations between items and time in user interaction sequences.Then,the same time sampling method is presented in the self-attention layer to learn the time correlations between items.Finally,in the fine-tuning stage,user interaction sequences are amplified from the time dimension to alleviate the data sparsity.Experiment results on real datasets show that the proposed TPTS-Rec model outperforms the baseline models.

Key words: Sequence recommendation, Pre-training, Self-supervised learning, Mutual information maximization, Time attribute

中图分类号: 

  • TP391
[1]LI L H,WEI C,LANGFORD J,et al.A Contextual-Bandit Approach to Personalized News Article Recommendation[C]//19th International Conference on World Wide Web.Association for Computing Machinery,2010:661-670.
[2]CHENG Z T,ZHONG T,ZHANG C M,et al.Survey of Recom-mender Systems Based on Graph Learning[J].Computer Science,2022,49(9):1-13.
[3]TIAN Z,PAN L M,YIN P,et al.Recommended algorithm for depth matrix decomposition[J].Journal of Software,2021,32(12):3917-3928.
[4]ZHOU F Q,CHENG W Q.Sequence Recommendation Based on Global Enhanced Graph Neural Network[J].Computer Science,2022,49(9):55-63.
[5]MOONEY R J,ROY L.Content-Based Book RecommendingUsing Learning for Text Categorization[C]//5th ACM Confe-rence on Digital Libraries.Association for Computing Machinery,2000:195-204.
[6]CHEN X S,LIU D,LEI C Y,et al.BERT4SessRec:Content-Based Video Relevance Prediction with Bidirectional Encoder Representations from Transformer[C]//27th ACM Interna-tional Conference on Multimedia.Association for Computing Machinery,2019:2597-2601.
[7]ZHOU K,ZHAO X W,BIAN S X,et al.Improving Conversational Recommender Systems via Knowledge Graph based Semantic Fusion[C]//26th ACM SIGKDD International Confe-rence on Knowledge Discovery & Data Mining.Association for Computing Machinery,2020:1006-1014.
[8]ZHANG Y D,LI C Z,XIE X,et al.Geometric Disentangled Collaborative Filtering[C]//45th International ACM SIGIR Conference on Research and Development in Information Retrieval.Association for Computing Machinery,2022:80-90.
[9]ZHAO M H,WU L,LIANG Y L,et al.Investigating Accuracy-Novelty Performance for Graph-based Collaborative Filtering[C]//45th International ACM SIGIR Conference on Research and Development in Information Retrieval.Association for Computing Machinery,2022:50-59.
[10]ZHOU K,WANG H,ZHAO X W,et al.S3-Rec:Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization[C]//29th ACM International Confe-rence on Information & Knowledge Management.Association for Computing Machinery,2020:1893-1902.
[11]CHEN Y J,LIU Z W,LI J,et al.Intent Contrastive Learning for Sequential Recommendation[C]//ACM Web Conference 2022.Association for Computing Machinery,2022:2172-2182.
[12]ZHOU K,YU H,ZHAO X W,et al.Filter-enhanced MLP is All You Need for Sequential Recommendation[C]//ACM Web Conference 2022.Association for Computing Machinery,2022:2388-2399.
[13]HIDASI B,TIKK D.General factorization framework for con-text-aware recommendations[J].arXiv:1401.4529,2014.
[14]FANG Y,TAN Z,CHEN Z Y,et al.Comparative meta-learning of heterogeneous information networks for cold start recommendation[J].Journal of Software,2023,34(10):1-19.
[15]WU S,TANG S,ZHU Y Y,et al.Session-based Recommendation with Graph Neural Networks[C]//Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence.AAAI Press,2019:346-353.
[16]SHAO Y W,ZHANG M,MA W Z,et al.Personalized recommendation method integrating the discovery of potential complementarity of commodities[J].Journal of Software,2020,31(4):1090-1100.
[17]DEVLIN J,CHANG M W,LEE K,et al.BERT:Pre-training of Deep Bidirectional Transformers for Language Understanding[J].arXiv:1810.04805,2018.
[18]LIAN D F,WU Y J,GE Y,et al.Geography-Aware Sequential Location Recommendation[C]//26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.Association for Computing Machinery,2020:2009-2019.
[19]BREESE J S,HECKERMAN D,KADIE C.Empirical analysisof predictive algorithms for collaborative filtering[C]//Fourteenth Conference on Uncertainty in Artificial Intelligence.Morgan Kaufmann Publishers Inc,1998:43-52.
[20]FANG H,ZHANG D,SHU Y H,et al.Deep Learning for Sequential Recommendation:Algorithms,Influential Factors,and Evaluations[J].ACM Transactions on Information Systems,2019,39(1):1-42.
[21]CHANG J X,GAO C,ZHENG Y,et al.Sequential Recommen-dation with Graph Neural Networks[C]//44th International ACM SIGIR Conference on Research and Development in Information Retrieval.Association for Computing Machinery,2021:378-387.
[22]YANG D Q,FANKHAUSER B,ROSSO P,et al.Location Prediction over Sparse User Mobility Traces Using RNNs:Flashback in Hidden States! [C]//Twenty-Ninth International Joint Conference on Artificial Intelligence and Seventeenth Pacific Rim International Conference on Artificial Intelligence.2020:2156-2161.
[23]SUN F,LIU J,WU J,et al.BERT4Rec:Sequential Recommendation with Bidirectional Encoder Representations from Transformer[C]//28th ACM International Conference on Information and Knowledge Management.Association for Computing Machinery,2019:1441-1450.
[24]ZHANG T T,ZHAO P P,LIU Y C,et al.Feature-level Deeper Self-Attention Network for Sequential Recommendation[C]//Twenty-Eighth International Joint Conference on Artificial Intelligence.AAAI Press,2019:4320-4326.
[25]HJELM R D,FEDOROV A,LAVOIE-MARCHILDON S,et al.Learning deep representations by mutual information estimation and maximization[C]//ICLR 2019 Conference Blind Submission(IJCAI-19).ICLR,2019:1-24.
[26]HE X N,DENG K,WANG X,et al.LightGCN:Simplifying and Powering Graph Convolution Network forRecommendation[C]//The 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval(SIGIR'20).Association for Computing Machinery,2020:639-648.
[27]HIDASI B,QUADRANA M,KARATZOGLOU A,et al.Parallel Recurrent Neural Network Architectures for Feature-rich Session-based Recommendations[C]//10th ACM Conference on Recommender Systems.Association for Computing Machinery,2016:241-248.
[28]TANG J,WANG K.Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding[C]//Eleventh ACM International Conference on Web Search and Data Mi-ning.Association for Computing Machinery,2018:565-573.
[29]YUE Y,HONG S A,GUESTRIN C.Hierarchical Exploration for Accelerating ContextualBandits[C]//29th International Coference on International Conference on Machine Learning(ICML'12).2012:1-8.
[30]YAO T S,YI X Y,CHENG Z D,et al.Self-supervised Learning for Deep Models in Recommendations[J].arXiv:2007.12865,2020.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发