Computer Science

Computer Science ›› 2024, Vol. 51 ›› Issue (5): 45-53.doi: 10.11896/jsjkx.230200049

• Database & Big Data & Data Science • Previous Articles     Next Articles

Time-aware Pre-training Method for Sequence Recommendation

CHEN Wenzhong1, CHEN Hongmei1,2, ZHOU Lihua1,2, FANG Yuan3   

  1. 1 School of Information Science and Engineering,Yunnan University,Kunming 650500,China
    2 Yunnan Key Laboratory of Intelligent Systems and Computing,Yunnan University,Kunming 650500,China
    3 South-Western Institute for Astronomy Research,Yunnan University,Kunming 650500,China
  • Received:2023-02-08 Revised:2023-05-24 Online:2024-05-15 Published:2024-05-08
  • About author:CHEN Wenzhong,born in 1994,postgraduate,is a member of CCF(No.J6048G).His main research interests include recommendation system and heterogeneous information network analysis.
    CHEN Hongmei,born in 1976,Ph.D,associate professor,is a member of CCF(No.49450M).Her main research interests include spatial data mining and location-based social network analysis.
  • Supported by:
    National Natural Science Foundation of China(62266050,62276227),Program for Young and Middle-aged Academic and Technical Reserve Leaders of Yunnan Province(202205AC160033),Yunnan Fundamental Research Projects(202201AS070015) and Open Project Program of Yunnan Key Laboratory of Intelligent Systems and Computing(ISC22Z02).

PDF (PC)

350

Abstract: Sequence recommendation aims to learn users' dynamic preferences and recommend the next items which users may be interested in by analyzing historical interaction sequences between users and items.The pre-training model has attracted attentions from researchers in sequence recommendation due to its advantage of being adapted for downstream tasks.The existing pre-training methods for sequence recommendation ignore the impact of time on user interaction behaviors in real life.To better capture the time semantics of interactions between users and items,this paper proposes a novel model TPTS-Rec(time-aware pre-training method for sequence recommendation).First,the time embedding matrix is introduced in the embedding layer to obtain the correlations between items and time in user interaction sequences.Then,the same time sampling method is presented in the self-attention layer to learn the time correlations between items.Finally,in the fine-tuning stage,user interaction sequences are amplified from the time dimension to alleviate the data sparsity.Experiment results on real datasets show that the proposed TPTS-Rec model outperforms the baseline models.

Key words: Sequence recommendation, Pre-training, Self-supervised learning, Mutual information maximization, Time attribute

CLC Number: 

  • TP391
[1]LI L H,WEI C,LANGFORD J,et al.A Contextual-Bandit Approach to Personalized News Article Recommendation[C]//19th International Conference on World Wide Web.Association for Computing Machinery,2010:661-670.
[2]CHENG Z T,ZHONG T,ZHANG C M,et al.Survey of Recom-mender Systems Based on Graph Learning[J].Computer Science,2022,49(9):1-13.
[3]TIAN Z,PAN L M,YIN P,et al.Recommended algorithm for depth matrix decomposition[J].Journal of Software,2021,32(12):3917-3928.
[4]ZHOU F Q,CHENG W Q.Sequence Recommendation Based on Global Enhanced Graph Neural Network[J].Computer Science,2022,49(9):55-63.
[5]MOONEY R J,ROY L.Content-Based Book RecommendingUsing Learning for Text Categorization[C]//5th ACM Confe-rence on Digital Libraries.Association for Computing Machinery,2000:195-204.
[6]CHEN X S,LIU D,LEI C Y,et al.BERT4SessRec:Content-Based Video Relevance Prediction with Bidirectional Encoder Representations from Transformer[C]//27th ACM Interna-tional Conference on Multimedia.Association for Computing Machinery,2019:2597-2601.
[7]ZHOU K,ZHAO X W,BIAN S X,et al.Improving Conversational Recommender Systems via Knowledge Graph based Semantic Fusion[C]//26th ACM SIGKDD International Confe-rence on Knowledge Discovery & Data Mining.Association for Computing Machinery,2020:1006-1014.
[8]ZHANG Y D,LI C Z,XIE X,et al.Geometric Disentangled Collaborative Filtering[C]//45th International ACM SIGIR Conference on Research and Development in Information Retrieval.Association for Computing Machinery,2022:80-90.
[9]ZHAO M H,WU L,LIANG Y L,et al.Investigating Accuracy-Novelty Performance for Graph-based Collaborative Filtering[C]//45th International ACM SIGIR Conference on Research and Development in Information Retrieval.Association for Computing Machinery,2022:50-59.
[10]ZHOU K,WANG H,ZHAO X W,et al.S3-Rec:Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization[C]//29th ACM International Confe-rence on Information & Knowledge Management.Association for Computing Machinery,2020:1893-1902.
[11]CHEN Y J,LIU Z W,LI J,et al.Intent Contrastive Learning for Sequential Recommendation[C]//ACM Web Conference 2022.Association for Computing Machinery,2022:2172-2182.
[12]ZHOU K,YU H,ZHAO X W,et al.Filter-enhanced MLP is All You Need for Sequential Recommendation[C]//ACM Web Conference 2022.Association for Computing Machinery,2022:2388-2399.
[13]HIDASI B,TIKK D.General factorization framework for con-text-aware recommendations[J].arXiv:1401.4529,2014.
[14]FANG Y,TAN Z,CHEN Z Y,et al.Comparative meta-learning of heterogeneous information networks for cold start recommendation[J].Journal of Software,2023,34(10):1-19.
[15]WU S,TANG S,ZHU Y Y,et al.Session-based Recommendation with Graph Neural Networks[C]//Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence.AAAI Press,2019:346-353.
[16]SHAO Y W,ZHANG M,MA W Z,et al.Personalized recommendation method integrating the discovery of potential complementarity of commodities[J].Journal of Software,2020,31(4):1090-1100.
[17]DEVLIN J,CHANG M W,LEE K,et al.BERT:Pre-training of Deep Bidirectional Transformers for Language Understanding[J].arXiv:1810.04805,2018.
[18]LIAN D F,WU Y J,GE Y,et al.Geography-Aware Sequential Location Recommendation[C]//26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.Association for Computing Machinery,2020:2009-2019.
[19]BREESE J S,HECKERMAN D,KADIE C.Empirical analysisof predictive algorithms for collaborative filtering[C]//Fourteenth Conference on Uncertainty in Artificial Intelligence.Morgan Kaufmann Publishers Inc,1998:43-52.
[20]FANG H,ZHANG D,SHU Y H,et al.Deep Learning for Sequential Recommendation:Algorithms,Influential Factors,and Evaluations[J].ACM Transactions on Information Systems,2019,39(1):1-42.
[21]CHANG J X,GAO C,ZHENG Y,et al.Sequential Recommen-dation with Graph Neural Networks[C]//44th International ACM SIGIR Conference on Research and Development in Information Retrieval.Association for Computing Machinery,2021:378-387.
[22]YANG D Q,FANKHAUSER B,ROSSO P,et al.Location Prediction over Sparse User Mobility Traces Using RNNs:Flashback in Hidden States! [C]//Twenty-Ninth International Joint Conference on Artificial Intelligence and Seventeenth Pacific Rim International Conference on Artificial Intelligence.2020:2156-2161.
[23]SUN F,LIU J,WU J,et al.BERT4Rec:Sequential Recommendation with Bidirectional Encoder Representations from Transformer[C]//28th ACM International Conference on Information and Knowledge Management.Association for Computing Machinery,2019:1441-1450.
[24]ZHANG T T,ZHAO P P,LIU Y C,et al.Feature-level Deeper Self-Attention Network for Sequential Recommendation[C]//Twenty-Eighth International Joint Conference on Artificial Intelligence.AAAI Press,2019:4320-4326.
[25]HJELM R D,FEDOROV A,LAVOIE-MARCHILDON S,et al.Learning deep representations by mutual information estimation and maximization[C]//ICLR 2019 Conference Blind Submission(IJCAI-19).ICLR,2019:1-24.
[26]HE X N,DENG K,WANG X,et al.LightGCN:Simplifying and Powering Graph Convolution Network forRecommendation[C]//The 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval(SIGIR'20).Association for Computing Machinery,2020:639-648.
[27]HIDASI B,QUADRANA M,KARATZOGLOU A,et al.Parallel Recurrent Neural Network Architectures for Feature-rich Session-based Recommendations[C]//10th ACM Conference on Recommender Systems.Association for Computing Machinery,2016:241-248.
[28]TANG J,WANG K.Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding[C]//Eleventh ACM International Conference on Web Search and Data Mi-ning.Association for Computing Machinery,2018:565-573.
[29]YUE Y,HONG S A,GUESTRIN C.Hierarchical Exploration for Accelerating ContextualBandits[C]//29th International Coference on International Conference on Machine Learning(ICML'12).2012:1-8.
[30]YAO T S,YI X Y,CHENG Z D,et al.Self-supervised Learning for Deep Models in Recommendations[J].arXiv:2007.12865,2020.
[1] ZHANG Zhiyuan, ZHANG Weiyan, SONG Yuqiu, RUAN Tong. Multilingual Event Detection Based on Cross-level and Multi-view Features Fusion [J]. Computer Science, 2024, 51(5): 208-215.
[2] WU Jiawei, FANG Quan, HU Jun, QIAN Shengsheng. Pre-training of Heterogeneous Graph Neural Networks for Multi-label Document Classification [J]. Computer Science, 2024, 51(1): 143-149.
[3] TANG Jia, GUO Yan, YE Mingwei, WU Guixing. Multimodal Pre-training Method for Multi-view Contrastive Learning and Semantic Enhancement [J]. Computer Science, 2024, 51(1): 168-174.
[4] LI Xiang, FAN Zhiguang, LIN Nan, CAO Yangjie, LI Xuexiang. Self-supervised Learning for 3D Real-scenes Question Answering [J]. Computer Science, 2023, 50(9): 220-226.
[5] YI Liu, GENG Xinyu, BAI Jing. Hierarchical Multi-label Text Classification Algorithm Based on Parallel Convolutional Network Information Fusion [J]. Computer Science, 2023, 50(9): 278-286.
[6] CAI Haoran, YANG Jian, YANG Lin, LIU Cong. Low-resource Thai Speech Synthesis Based on Alternate Training and Pre-training [J]. Computer Science, 2023, 50(6A): 220800127-5.
[7] ZENG Wu, MAO Guojun. Few-shot Learning Method Based on Multi-graph Feature Aggregation [J]. Computer Science, 2023, 50(6A): 220400029-10.
[8] WANG Taiyan, PAN Zulie, YU Lu, SONG Jingbin. Binary Code Similarity Detection Method Based on Pre-training Assembly Instruction Representation [J]. Computer Science, 2023, 50(4): 288-297.
[9] LUO Xiaohui, WU Yun, WANG Chenxing, YU Wenting. Sequential Recommendation Model Based on User’s Long and Short Term Preference [J]. Computer Science, 2023, 50(4): 47-55.
[10] LIU Zhe, YIN Chengfeng, LI Tianrui. Chinese Spelling Check Based on BERT and Multi-feature Fusion Embedding [J]. Computer Science, 2023, 50(3): 282-290.
[11] WANG Pengyu, TAI Wenxin, LIU Fang, ZHONG Ting, LUO Xucheng, ZHOU Fan. Self-supervised Flight Trajectory Prediction Based on Data Augmentation [J]. Computer Science, 2023, 50(2): 130-137.
[12] ZHU Lei, WANG Shanmin, LIU Qingshan. Self-supervised 3D Face Reconstruction Based on Detailed Face Mask [J]. Computer Science, 2023, 50(2): 214-220.
[13] SU Qi, WANG Hongling, WANG Zhongqing. Unsupervised Script Summarization Based on Pre-trained Model [J]. Computer Science, 2023, 50(2): 310-316.
[14] HE Wenhao, WU Chunjiang, ZHOU Shijie, HE Chaoxin. Study on Short Text Clustering with Unsupervised SimCSE [J]. Computer Science, 2023, 50(11): 71-76.
[15] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.