面向大语言模型的推荐系统综述

doi:10.11896/jsjkx.240800111

摘要/Abstract

摘要： 大型语言模型已成为自然语言处理领域中的有力使用工具,并且在近期也成功吸引了推荐系统领域的广泛关注。这些模型凭借自监督学习在庞大的数据集上进行了深度训练,从而在通用表征学习上取得了显著成果。通过微调、提示调整等高效的迁移技术,它们有望全面提升推荐系统的各项性能。在利用语言模型的力量来优化推荐质量的过程中,关键在于充分利用其高质量的文本特征表征以及广泛的外部知识库,以此为基础构建物品与用户之间的紧密联系。为了全面而深入地理解当前基于大型语言模型的推荐系统,文中将这些模型细致地划分为两大类:用于推荐的判别式大型语言模型和用于推荐的生成式大型语言模型。同时,对于后者,还进一步将其细分为约束生成和自由生成,并对这两种方法的相关研究进行了详尽的总结。此外,还指出了该领域面临的关键挑战,并分享了一些有价值的发现,希望可以为研究人员和从业者提供宝贵的灵感与启示。

关键词: 大语言模型, 推荐系统, 自然语言处理

Abstract: Large language models have emerged as highly effective tools in the field of natural language processing(NLP) and have recently garnered considerable attention in the domain of recommendation systems(RS).These models have undergone extensive training on vast datasets through self-supervised learning,achieving remarkable results in learning universal representations.With efficient transfer techniques such as fine-tuning and prompt tuning,they have the potential to significantly enhance various aspects of recommendation system performance.The crux of leveraging language models to optimize recommendation quality lies in fully utilizing their high-quality text feature representations and extensive external knowledge bases to establish strong connections between items and users.To gain a comprehensive and in-depth understanding of current recommendation systems based on large language models,this paper meticulously categorizes these models into two main types:discriminative large language models for recommendation(DLLM4Rec) and generative large language models for recommendation(GLLM4Rec).Furthermore,the latter is subdivided into constrained generation and free generation,and a detailed summary of relevant research on these two approaches is provide.Additionally,this paper identifies key challenges in this field and shares valuable findings,hoping to provide researchers and practitioners with precious inspiration and insights.

Key words: Large language model, Recommender system, Natural language processing

中图分类号:

TP391

卡祖铭, 赵鹏, 张波, 傅晓宁. 面向大语言模型的推荐系统综述[J]. 计算机科学, 2024, 51(11A): 240800111-11. https://doi.org/10.11896/jsjkx.240800111

KA Zuming, ZHAO Peng, ZHANG Bo, FU Xiaoning. Survey of Recommender Systems for Large Language Models[J]. Computer Science, 2024, 51(11A): 240800111-11. https://doi.org/10.11896/jsjkx.240800111

参考文献

[1]GAO Y,SHENG T,XIANG Y Y,et al.Chat-rec:Towards interactive and explainable llms augmented recommender system[J].2003.
[2]CHEN J,MA L,LI X,et al.Knowledge grap hcompletion mo-dels are few-shot learners:An empirical study of relation labeling in e-commerce with llms[J].arXiv:2305.09858,2023.
[3]CHEN X,FAN W,CHEN J,et al.Fairly adaptive negative sampling for recommendations[C]//Proceedings of the ACM Web Conference.2023:3723-3733.
[4]QIN L,WU W S,LIU D,et al.Autonomous planning and processing framework for complex tasks based on large language models[J].Acta Automatica Sinica,2024,50(4):862-872.
[5]FAN W,ZHAO X,CHEN X,et al.A comprehensive survey on trustworthy recommender systems[J].arXiv:2209.10117,2022.
[6]HE X,DENG K,WANG X,et al.Lightgcn:Simplifying andpowering graph convolution network for recommendation[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval.2020:639-648.
[7]FAN W,DERR T,MA Y,et al.Deep adversarial social recommendation[C]//28th International Joint Conference on Artificial Intelligence(IJCAI-19).International Joint Conferences on Artificial Intelligence,2019:1351-1357.
[8]ZHENG L,NOROOZI V,YU P S.Joint deep modeling of usersand items using reviews for recommendation[C]//Proceedings of the tenth ACM International Conference on Web Search and Data Mining.2017:425-434.
[9]ZHANG S,YAO L,SUN A,et al.Deep learning based recommender system:A survey and new perspectives[J].ACM Computing Surveys(CSUR),2019,52(1):1-38.
[10]FAN W,LIU C,LIU Y,et al.Generative diffusion models on graphs:Methods and applications[J].arXiv:2302.02591,2023.
[11]HIDASI B,KARATZOGLOU A,BALTRUNAS L,et al.Session-based recommendations with recurrent neural networks[J].arXiv:1511.06939,2015.
[12]FAN W,MA Y,YIN D,et al.Deep social collaborative filtering[C]//Proceedings of the 13th ACM Conference on Recommender Systems.2019:305-313.
[13]FAN W,MA Y,LI Q,et al.Graph neural networks for social recommendation[C]//The World Wide Web Conference.2019:417-426.
[14]QIU Z,WU X,GAO J,et al.U-bert:Pre-training user representations for improved recommendation[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2021:4320-4327.
[15]BROWN T,MANN B,RYDER N,et al.Language models are few-shot learners[J].NeurIPS,2020.
[16]ZHOU L,PALANGI H,ZHANG L,et al.Unified vision-lan-guage pre-training for image captioning and vqa[C]//Procee-dings of the AAAI Conference on Artificial Intelligence.2020:13041-13049.
[17]LI J,LIU Y,FAN W,et al.Empowering molecule discovery for mole-cule-caption translationwith large language models:A chatgpt perspective[J].a-Xiv:2306.06615,2023.
[18]CHEN Z,MAO H,LI H,et al.Exploring the potential of large language models(llms) in learning on graphs[J].arXiv:2307.03393,2023.
[19]ZHAO W X,ZHOU K,LI J,et al.A survey of large language models[J].arXiv:2303.18223,2023.
[20]ZHANG J,XIE R,HOU Y,et al.Recommendation as instruction following:A large language model empowered recommendation approach[J].arXiv:2305.07001,2023.
[21]DEVLIN J,CHANG M W,LEE K,et al.Bert:Pre-training of deep b-idirectional transformers for language understanding[J].arXiv:1810.04805,2018.
[22]RADFORD A,NARASIMHAN K,SALIMANS T,et al. Improving Language Understanding by Generative Pre-Training [OL].https://openai.com/index/language-unsupervised/.
[23]RAFFEL C,SHAZEER N,ROBERTS A,et al.Exploring the limits of transfer learning with a unified text-to-text transformer[J].The Journal of Machine Learning Research,2020,21(1):5485-5551.
[24]VASWANI A,SHAZEER N,PARMAR N,et al.At-tention is all you need[J].Advances in Neural Information Processing Systems,2017,30:5998-6008.
[25]ZHANG Z,ZHANG G,HOU B,et al.Certified robustness for large language-models with self-denoising[J].arXiv:2307.07171,2023
[26]THOPPILAN R,DE FREITAS D,HALL J,et al.Lamda:Language models for dialog applications[J].arXiv:2201.08239,2022.
[27]CHOWDHERY A,NARANG S,DEVLIN J,et al.Palm:Scaling language modeling with pathways[J].arXiv:2204.02311,2022.
[28]CHANG W L,LI Z,LIN Z,et al.Vicuna:An Open-Source Chatbot Impressing GPT-4 with 90%^* ChatGPT Quality [C/OL]//https://vicuna.lmsys.org(accessed 14 April 2023).2023.
[29]KIM H J,CHO H,KIM J,et al.Self-generated in-context lear-ning:Leveraging auto-regressive language models as a demons-tration generator[J].arXiv:2206.08082,2022.
[30]RUBIN O,HERZIG J,BERANT J.Learning to retrieveprompts for incontext learning[J].arXiv:2112.08633,2021.
[31]WEI J,WANG X,SCHUURMANS D,et al.Ch-ain of thought prompting elicits reasoning in large language models[J].arXiv:2201.11903,2022.
[32]WANG X,WEI J,SCHUURMANS D,et al.Self-consistencyimproves c-hain of thought reasoning in language-models[J].arXiv:2203.11171,2022.
[33]ZELIKMAN E,WU Y,MU J,et al.Star:Bootstrapping reaso-ning with reasoning[J].Advances in Neural Information Proces-sing Systems,2022,35:15476-15488.
[34]FEI H,LI B,LIU Q,et al.Reasoning implicit sentiment withchain-of-thought prompting[J].arXiv:2305.11255,2023.
[35]JIN Z,LU W.Tab-cot:Zero-shot tabular chain of thought[J].arXiv:2305.17812,2023.
[36]KASNECI E,SEßLER K,KÜCHEMANN S,et al.Chatgpt for good? on opportunities and challenges of large language models for education[J].Learning and Individual Differences,2023,103:102274.
[37]WU S,IRSOY O,LU S,et al.Bloomberggpt:A large language model for finance[J].arXiv:2303.17564,2023.
[38]CUI Z,MA J,ZHOU C,et al.M6-rec:Generative pretrained language models are open-ended recommender systems[J].arXiv:2205.08084,2022.
[39]CUI Z Y,MA J X,ZHOU C,et al.M6-rec:Generative pretrainedlanguage models are open-ended recommender systems[J].ar-Xiv:2205.08084,2022.
[40]GENG S,LIU S,FU Z,et al.Rec-ommendation as language processing(rlp):A unified pretrain,personalized prompt & predict paradigm(p5)[C]//Proceedings of the 16th ACM Conference on Recommender Systems.2022:299-315.
[41]WANG X,ZHOU K,WEN J R,et al.To-wards unified conversational recommender systems via knowledge-enhanced prompt learning[C]//Proceedings of the 28th ACM SIGKDD Confe-rence on Knowledge Discovery and Data Mining.2022:1929-1937.
[42]DENG Y,ZHANG W,XU W,et al.A unified multi-task lear-ning framework for multi-goal conversational recommender systems[J].ACM T ransactions on Information Systems,2023,41(3):1-25.
[43] SHENG B,GUAN Z Y,LEE L L,et al.diabetes managementbased on the big language model:potential and prospect [J].Science Bulletin,2024,69(5):583-588.
[44]CHEN X,ZHANG Y F,QIN Z.Dy-namic explainable recommendation based on neural at-tentive models[C]//AAAI.2019:53-60.
[45]KANG W C,MCAULEY J L.Self-attentive sequential recom-mendation[C]//ICDM.IEEE,2018:197-206.
[46]ZHANG Y F,AI Q Y,CHEN X,et al.Joint representationlearning for top-n recommendation with heterogeneous information sources[C]//CIKM.2017:1449-1458.
[47]HE X N,LIAO L Z,ZHANG H W,et al.Neural Collaborative Filtering[C]//WWW.ACM,2017:173-182.
[48]SUN F,LIU J,WU J,et al.BERT4Rec:Sequen-tial recommendation with bidirectional encoder repre-sentations from transformer[C]//CIKM.ACM,2019:1441-1450.
[49]TANG J X,WANG K.Personalized top-n sequential recommendationvia convolutional sequence embedding[C]//WSDM.ACM,2018:565-573
[50]WANG X,HE X N,WANG M,et al.Neural Graph Collaborative Filtering[C]//SIGIR.ACM,2019:165-174.
[51]HE X N,DENG K,WANG X,et al.Lightgcn:Simpli-fying and powering graph convolution network for recommendation[C]//SIGIR.2020:639-648.
[52]DEVLIN J,CHANG M W,LEE K,et al.BERT:pre-training of deep bidirectional transformers for language understanding[C]//Association for Computational Linguistics.NAACL-HLT(1),2019:4171-4186.
[53]PENHA G,HAUFF C.What does BERT know about books,movies and mu-sic? probing BERT for conversational recommendation[C]//RecSys.ACM,2020:388-397.
[54]BROWN T B,MANN B,NICKRYDER,et al.Language modelsare fewshot learners[C]//NeurIPS.2020.
[55]OUYANG L,WU J,JIANG X,et al.Training lan-guage models to follow instructions with human feed-back[C]//NeurIPS.2022.
[56]LIU J L,LIU C,LV R J,et al.Is chatgpt a good recommender? A preliminary study[J].CoRR,abs/2304.10149,2023.
[57]DAI S B,SHAO N L,ZHAO H Y,et al.Uncovering chatgpt’s capabilities inrecom-mender systems[J].CoRR,abs/2305.02182,2023.
[58]DAI D M,SUN Y T,DONG L,et al.Why can GPT learn in-context? language models secretly perform gradient descent as meta-optimizers[J].CoRR,abs/2212.10559,2022.
[59]WANG W J,LIN X Y,FENG F L,et al.Generative recommendation:Towards next-generation recommender paradigm.CoRR[J].abs/2304.03516,2023.
[60] LIU Y Q,WANG Y,SUN L C,et al.Rec-GPT4V:Multimodal Recommendation with Large Vision-Language Models[J].arX-iv:2402.08670.2024.
[61]HUA W Y,XU S Y,GE Y Q,et al.How to I-ndex Item IDs for Recommendation Foundation Models[J].arXiv:2305.06569,2023.
[62]LI L,ZHANG Y F,LIU D G,et al.Large language models for generative recommendation:A survey and visionary discussions[J].arXiv:2309.01157,2023.
[63]BAO K Q,ZHANG J Z,WANG W J,et al.A bi-step grounding paradigm for large language models in recommendation systems[J].arXiv:2308.08434,2023.
[64]CHU Z X,HAO H Y,OUYANG X,et al.Leveraging largelanguage models for pre-trained recommender systems[J].ar-Xiv:2308.10837,2023.
[65]RAJPUT S,MEHTA N,SINGH A,et al.Recommender Systems with Generative Retrieval[C]//NeurIPS.Curran Asso-ciates,Inc,2023.
[66]LIN X Y,WANG W J,LI Y Q,et al.A multi-facet paradigm to bridge large language model and recom-mendation[J].arXiv:2310.06491,2023.
[67]WEI T X,JIN B W,LI R R,et al.Towards Universal Multi-Modal Personalization:A Language Model Empowered Generative Paradigm[C]//ICLR.2024.
[68]XU Y Y,WANG W J,FENG F X,et al..DiFashion:Towards Personalized Outfit Generation[C]//SIGIR.2024
[69]LI Z L,JI J C,GE Y Q,et al.PAP-REC:Personalized Auto-ma-tic Prompt for Recommendation Language Model[J].arXiv:2402.00284,2024.
[70]GENG S J,TAN J T,LIU S C,et al.VIP5:Towards Multimodal Foundation Models for Recommendation[C]//EMNLP.2023:9606-9620.
[71]ZHENG B W,HOU Y P,LU H Y,et al.Adapting large language models by integrating collabora-tivesemantics for recommendation[J].arXiv:2311.09049,2023.
[72]ZHANG Y H,DING H,SHUI Z R,et al.Language models as recommender systems:Evaluations and limi-tations[C]//2021.
[73]WANG L, LIM E P.Zero-Shot Next-Item Recommendationusing Large Pretrained Language Models[J].arXiv:2304.03153,2023.
[74]LUO S C,HE B W,ZHAO H H,et al.RecRanker:Instruction Tuning Large Language Model as Ranker for Top-k Recommendation[J].arXiv:2312.16018,2023.
[75]ZHANG J J,XIE R B,HOU Y P,et al.Recommendation as instruction following:A large language model empowered recommendation approach[J].arXiv:2305.07001,2023.
[76]LIU Y Q,WANG Y,SUN L C,et al.Rec-GPT4V:Multimodal Recommendation with Large Vision-Language Models[J].ar-Xiv:2402.08670,2024.
[77]GU Z,HE X,YU P,et al.Automatic quantitative stroke severity assessment based on Chinese clinical named entity recognition with domain adaptive pretrained large language model[J].Artificial Intelligence In Medicine,2024,150:102822.
[78] LI N.Exploration of the Application of Big Language Model in Financial Shared Center [J].China Agricultural Accounting,2024,34(6):88-90.
[79]LU M F.Research on the Application Principles,Challenges,and Implementation Paths of Large Language Models in the Financial Sector [J/OL].Journal of Chongqing Technology and Business University(Social Sciences Edition),1-13.[2400-04-08].
[80]LEE G G,LATIF E,WU X,et al.Applying large language mo-dels and chain-of-thought for automatic scoring[J].Computers and Education:Artificial Intelligence,2024,6:100213.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed