Computer Science ›› 2024, Vol. 51 ›› Issue (11A): 240800111-11.doi: 10.11896/jsjkx.240800111

• Intelligent Computing • Previous Articles     Next Articles

Survey of Recommender Systems for Large Language Models

KA Zuming, ZHAO Peng, ZHANG Bo, FU Xiaoning   

  1. College of Operational Support,Rocket Force University of Engineering,Xi'an 710025,China
  • Online:2024-11-16 Published:2024-11-13
  • About author:KA Zuming,born in 2000,postgra-duate.Her main research interests include recommendation systems and large language models.
    ZHAO Peng,born in 1979,Ph.D,asso-ciate professor.His main research intere-sts include intelligent information processing,recommendation systems,and distributed computing.

Abstract: Large language models have emerged as highly effective tools in the field of natural language processing(NLP) and have recently garnered considerable attention in the domain of recommendation systems(RS).These models have undergone extensive training on vast datasets through self-supervised learning,achieving remarkable results in learning universal representations.With efficient transfer techniques such as fine-tuning and prompt tuning,they have the potential to significantly enhance various aspects of recommendation system performance.The crux of leveraging language models to optimize recommendation quality lies in fully utilizing their high-quality text feature representations and extensive external knowledge bases to establish strong connections between items and users.To gain a comprehensive and in-depth understanding of current recommendation systems based on large language models,this paper meticulously categorizes these models into two main types:discriminative large language models for recommendation(DLLM4Rec) and generative large language models for recommendation(GLLM4Rec).Furthermore,the latter is subdivided into constrained generation and free generation,and a detailed summary of relevant research on these two approaches is provide.Additionally,this paper identifies key challenges in this field and shares valuable findings,hoping to provide researchers and practitioners with precious inspiration and insights.

Key words: Large language model, Recommender system, Natural language processing

CLC Number: 

  • TP391
[1]GAO Y,SHENG T,XIANG Y Y,et al.Chat-rec:Towards interactive and explainable llms augmented recommender system[J].2003.
[2]CHEN J,MA L,LI X,et al.Knowledge grap hcompletion mo-dels are few-shot learners:An empirical study of relation labeling in e-commerce with llms[J].arXiv:2305.09858,2023.
[3]CHEN X,FAN W,CHEN J,et al.Fairly adaptive negative sampling for recommendations[C]//Proceedings of the ACM Web Conference.2023:3723-3733.
[4]QIN L,WU W S,LIU D,et al.Autonomous planning and processing framework for complex tasks based on large language models[J].Acta Automatica Sinica,2024,50(4):862-872.
[5]FAN W,ZHAO X,CHEN X,et al.A comprehensive survey on trustworthy recommender systems[J].arXiv:2209.10117,2022.
[6]HE X,DENG K,WANG X,et al.Lightgcn:Simplifying andpowering graph convolution network for recommendation[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval.2020:639-648.
[7]FAN W,DERR T,MA Y,et al.Deep adversarial social recommendation[C]//28th International Joint Conference on Artificial Intelligence(IJCAI-19).International Joint Conferences on Artificial Intelligence,2019:1351-1357.
[8]ZHENG L,NOROOZI V,YU P S.Joint deep modeling of usersand items using reviews for recommendation[C]//Proceedings of the tenth ACM International Conference on Web Search and Data Mining.2017:425-434.
[9]ZHANG S,YAO L,SUN A,et al.Deep learning based recommender system:A survey and new perspectives[J].ACM Computing Surveys(CSUR),2019,52(1):1-38.
[10]FAN W,LIU C,LIU Y,et al.Generative diffusion models on graphs:Methods and applications[J].arXiv:2302.02591,2023.
[11]HIDASI B,KARATZOGLOU A,BALTRUNAS L,et al.Session-based recommendations with recurrent neural networks[J].arXiv:1511.06939,2015.
[12]FAN W,MA Y,YIN D,et al.Deep social collaborative filtering[C]//Proceedings of the 13th ACM Conference on Recommender Systems.2019:305-313.
[13]FAN W,MA Y,LI Q,et al.Graph neural networks for social recommendation[C]//The World Wide Web Conference.2019:417-426.
[14]QIU Z,WU X,GAO J,et al.U-bert:Pre-training user representations for improved recommendation[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2021:4320-4327.
[15]BROWN T,MANN B,RYDER N,et al.Language models are few-shot learners[J].NeurIPS,2020.
[16]ZHOU L,PALANGI H,ZHANG L,et al.Unified vision-lan-guage pre-training for image captioning and vqa[C]//Procee-dings of the AAAI Conference on Artificial Intelligence.2020:13041-13049.
[17]LI J,LIU Y,FAN W,et al.Empowering molecule discovery for mole-cule-caption translationwith large language models:A chatgpt perspective[J].a-Xiv:2306.06615,2023.
[18]CHEN Z,MAO H,LI H,et al.Exploring the potential of large language models(llms) in learning on graphs[J].arXiv:2307.03393,2023.
[19]ZHAO W X,ZHOU K,LI J,et al.A survey of large language models[J].arXiv:2303.18223,2023.
[20]ZHANG J,XIE R,HOU Y,et al.Recommendation as instruction following:A large language model empowered recommendation approach[J].arXiv:2305.07001,2023.
[21]DEVLIN J,CHANG M W,LEE K,et al.Bert:Pre-training of deep b-idirectional transformers for language understanding[J].arXiv:1810.04805,2018.
[22]RADFORD A,NARASIMHAN K,SALIMANS T,et al. Improving Language Understanding by Generative Pre-Training [OL].https://openai.com/index/language-unsupervised/.
[23]RAFFEL C,SHAZEER N,ROBERTS A,et al.Exploring the limits of transfer learning with a unified text-to-text transformer[J].The Journal of Machine Learning Research,2020,21(1):5485-5551.
[24]VASWANI A,SHAZEER N,PARMAR N,et al.At-tention is all you need[J].Advances in Neural Information Processing Systems,2017,30:5998-6008.
[25]ZHANG Z,ZHANG G,HOU B,et al.Certified robustness for large language-models with self-denoising[J].arXiv:2307.07171,2023
[26]THOPPILAN R,DE FREITAS D,HALL J,et al.Lamda:Language models for dialog applications[J].arXiv:2201.08239,2022.
[27]CHOWDHERY A,NARANG S,DEVLIN J,et al.Palm:Scaling language modeling with pathways[J].arXiv:2204.02311,2022.
[28]CHANG W L,LI Z,LIN Z,et al.Vicuna:An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Quality [C/OL]//https://vicuna.lmsys.org(accessed 14 April 2023).2023.
[29]KIM H J,CHO H,KIM J,et al.Self-generated in-context lear-ning:Leveraging auto-regressive language models as a demons-tration generator[J].arXiv:2206.08082,2022.
[30]RUBIN O,HERZIG J,BERANT J.Learning to retrieveprompts for incontext learning[J].arXiv:2112.08633,2021.
[31]WEI J,WANG X,SCHUURMANS D,et al.Ch-ain of thought prompting elicits reasoning in large language models[J].arXiv:2201.11903,2022.
[32]WANG X,WEI J,SCHUURMANS D,et al.Self-consistencyimproves c-hain of thought reasoning in language-models[J].arXiv:2203.11171,2022.
[33]ZELIKMAN E,WU Y,MU J,et al.Star:Bootstrapping reaso-ning with reasoning[J].Advances in Neural Information Proces-sing Systems,2022,35:15476-15488.
[34]FEI H,LI B,LIU Q,et al.Reasoning implicit sentiment withchain-of-thought prompting[J].arXiv:2305.11255,2023.
[35]JIN Z,LU W.Tab-cot:Zero-shot tabular chain of thought[J].arXiv:2305.17812,2023.
[36]KASNECI E,SEßLER K,KÜCHEMANN S,et al.Chatgpt for good? on opportunities and challenges of large language models for education[J].Learning and Individual Differences,2023,103:102274.
[37]WU S,IRSOY O,LU S,et al.Bloomberggpt:A large language model for finance[J].arXiv:2303.17564,2023.
[38]CUI Z,MA J,ZHOU C,et al.M6-rec:Generative pretrained language models are open-ended recommender systems[J].arXiv:2205.08084,2022.
[39]CUI Z Y,MA J X,ZHOU C,et al.M6-rec:Generative pretrainedlanguage models are open-ended recommender systems[J].ar-Xiv:2205.08084,2022.
[40]GENG S,LIU S,FU Z,et al.Rec-ommendation as language processing(rlp):A unified pretrain,personalized prompt & predict paradigm(p5)[C]//Proceedings of the 16th ACM Conference on Recommender Systems.2022:299-315.
[41]WANG X,ZHOU K,WEN J R,et al.To-wards unified conversational recommender systems via knowledge-enhanced prompt learning[C]//Proceedings of the 28th ACM SIGKDD Confe-rence on Knowledge Discovery and Data Mining.2022:1929-1937.
[42]DENG Y,ZHANG W,XU W,et al.A unified multi-task lear-ning framework for multi-goal conversational recommender systems[J].ACM T ransactions on Information Systems,2023,41(3):1-25.
[43] SHENG B,GUAN Z Y,LEE L L,et al.diabetes managementbased on the big language model:potential and prospect [J].Science Bulletin,2024,69(5):583-588.
[44]CHEN X,ZHANG Y F,QIN Z.Dy-namic explainable recommendation based on neural at-tentive models[C]//AAAI.2019:53-60.
[45]KANG W C,MCAULEY J L.Self-attentive sequential recom-mendation[C]//ICDM.IEEE,2018:197-206.
[46]ZHANG Y F,AI Q Y,CHEN X,et al.Joint representationlearning for top-n recommendation with heterogeneous information sources[C]//CIKM.2017:1449-1458.
[47]HE X N,LIAO L Z,ZHANG H W,et al.Neural Collaborative Filtering[C]//WWW.ACM,2017:173-182.
[48]SUN F,LIU J,WU J,et al.BERT4Rec:Sequen-tial recommendation with bidirectional encoder repre-sentations from transformer[C]//CIKM.ACM,2019:1441-1450.
[49]TANG J X,WANG K.Personalized top-n sequential recommendationvia convolutional sequence embedding[C]//WSDM.ACM,2018:565-573
[50]WANG X,HE X N,WANG M,et al.Neural Graph Collaborative Filtering[C]//SIGIR.ACM,2019:165-174.
[51]HE X N,DENG K,WANG X,et al.Lightgcn:Simpli-fying and powering graph convolution network for recommendation[C]//SIGIR.2020:639-648.
[52]DEVLIN J,CHANG M W,LEE K,et al.BERT:pre-training of deep bidirectional transformers for language understanding[C]//Association for Computational Linguistics.NAACL-HLT(1),2019:4171-4186.
[53]PENHA G,HAUFF C.What does BERT know about books,movies and mu-sic? probing BERT for conversational recommendation[C]//RecSys.ACM,2020:388-397.
[54]BROWN T B,MANN B,NICKRYDER,et al.Language modelsare fewshot learners[C]//NeurIPS.2020.
[55]OUYANG L,WU J,JIANG X,et al.Training lan-guage models to follow instructions with human feed-back[C]//NeurIPS.2022.
[56]LIU J L,LIU C,LV R J,et al.Is chatgpt a good recommender? A preliminary study[J].CoRR,abs/2304.10149,2023.
[57]DAI S B,SHAO N L,ZHAO H Y,et al.Uncovering chatgpt’s capabilities inrecom-mender systems[J].CoRR,abs/2305.02182,2023.
[58]DAI D M,SUN Y T,DONG L,et al.Why can GPT learn in-context? language models secretly perform gradient descent as meta-optimizers[J].CoRR,abs/2212.10559,2022.
[59]WANG W J,LIN X Y,FENG F L,et al.Generative recommendation:Towards next-generation recommender paradigm.CoRR[J].abs/2304.03516,2023.
[60] LIU Y Q,WANG Y,SUN L C,et al.Rec-GPT4V:Multimodal Recommendation with Large Vision-Language Models[J].arX-iv:2402.08670.2024.
[61]HUA W Y,XU S Y,GE Y Q,et al.How to I-ndex Item IDs for Recommendation Foundation Models[J].arXiv:2305.06569,2023.
[62]LI L,ZHANG Y F,LIU D G,et al.Large language models for generative recommendation:A survey and visionary discussions[J].arXiv:2309.01157,2023.
[63]BAO K Q,ZHANG J Z,WANG W J,et al.A bi-step grounding paradigm for large language models in recommendation systems[J].arXiv:2308.08434,2023.
[64]CHU Z X,HAO H Y,OUYANG X,et al.Leveraging largelanguage models for pre-trained recommender systems[J].ar-Xiv:2308.10837,2023.
[65]RAJPUT S,MEHTA N,SINGH A,et al.Recommender Systems with Generative Retrieval[C]//NeurIPS.Curran Asso-ciates,Inc,2023.
[66]LIN X Y,WANG W J,LI Y Q,et al.A multi-facet paradigm to bridge large language model and recom-mendation[J].arXiv:2310.06491,2023.
[67]WEI T X,JIN B W,LI R R,et al.Towards Universal Multi-Modal Personalization:A Language Model Empowered Generative Paradigm[C]//ICLR.2024.
[68]XU Y Y,WANG W J,FENG F X,et al..DiFashion:Towards Personalized Outfit Generation[C]//SIGIR.2024
[69]LI Z L,JI J C,GE Y Q,et al.PAP-REC:Personalized Auto-ma-tic Prompt for Recommendation Language Model[J].arXiv:2402.00284,2024.
[70]GENG S J,TAN J T,LIU S C,et al.VIP5:Towards Multimodal Foundation Models for Recommendation[C]//EMNLP.2023:9606-9620.
[71]ZHENG B W,HOU Y P,LU H Y,et al.Adapting large language models by integrating collabora-tivesemantics for recommendation[J].arXiv:2311.09049,2023.
[72]ZHANG Y H,DING H,SHUI Z R,et al.Language models as recommender systems:Evaluations and limi-tations[C]//2021.
[73]WANG L, LIM E P.Zero-Shot Next-Item Recommendationusing Large Pretrained Language Models[J].arXiv:2304.03153,2023.
[74]LUO S C,HE B W,ZHAO H H,et al.RecRanker:Instruction Tuning Large Language Model as Ranker for Top-k Recommendation[J].arXiv:2312.16018,2023.
[75]ZHANG J J,XIE R B,HOU Y P,et al.Recommendation as instruction following:A large language model empowered recommendation approach[J].arXiv:2305.07001,2023.
[76]LIU Y Q,WANG Y,SUN L C,et al.Rec-GPT4V:Multimodal Recommendation with Large Vision-Language Models[J].ar-Xiv:2402.08670,2024.
[77]GU Z,HE X,YU P,et al.Automatic quantitative stroke severity assessment based on Chinese clinical named entity recognition with domain adaptive pretrained large language model[J].Artificial Intelligence In Medicine,2024,150:102822.
[78] LI N.Exploration of the Application of Big Language Model in Financial Shared Center [J].China Agricultural Accounting,2024,34(6):88-90.
[79]LU M F.Research on the Application Principles,Challenges,and Implementation Paths of Large Language Models in the Financial Sector [J/OL].Journal of Chongqing Technology and Business University(Social Sciences Edition),1-13.[2400-04-08].
[80]LEE G G,LATIF E,WU X,et al.Applying large language mo-dels and chain-of-thought for automatic scoring[J].Computers and Education:Artificial Intelligence,2024,6:100213.
[1] GUO Zhiqiang, GUAN Donghai, YUAN Weiwei. Word-Character Model with Low Lexical Information Loss for Chinese NER [J]. Computer Science, 2024, 51(8): 272-280.
[2] LIU Yumeng, ZHAO Yijing, WANG Bicong, WANG Chao, ZHANG Baomin. Advances in SQL Intelligent Synthesis Technology [J]. Computer Science, 2024, 51(7): 40-48.
[3] YANG Zhenzhen, WANG Dongtao, YANG Yongpeng, HUA Renyu. Multi-embedding Fusion Based on top-N Recommendation [J]. Computer Science, 2024, 51(7): 140-145.
[4] YANG Binxia, LUO Xudong, SUN Kaili. Recent Progress on Machine Translation Based on Pre-trained Language Models [J]. Computer Science, 2024, 51(6A): 230700112-8.
[5] WANG Yingjie, ZHANG Chengye, BAI Fengbo, WANG Zumin. Named Entity Recognition Approach of Judicial Documents Based on Transformer [J]. Computer Science, 2024, 51(6A): 230500164-9.
[6] LI Minzhe, YIN Jibin. TCM Named Entity Recognition Model Combining BERT Model and Lexical Enhancement [J]. Computer Science, 2024, 51(6A): 230900030-6.
[7] PENG Bo, LI Yaodong, GONG Xianfu, LI Hao. Method for Entity Relation Extraction Based on Heterogeneous Graph Neural Networks and TextSemantic Enhancement [J]. Computer Science, 2024, 51(6A): 230700071-5.
[8] LI Bin, WANG Haochang. Implementation and Application of Chinese Grammatical Error Diagnosis System Based on CRF [J]. Computer Science, 2024, 51(6A): 230900073-6.
[9] ZHANG Mingdao, ZHOU Xin, WU Xiaohong, QING Linbo, HE Xiaohai. Unified Fake News Detection Based on Semantic Expansion and HDGCN [J]. Computer Science, 2024, 51(4): 299-306.
[10] TU Xin, ZHANG Wei, LI Jidong, LI Meijiao , LONG Xiangbo. Study on Automatic Classification of English Tense Exercises for Intelligent Online Teaching [J]. Computer Science, 2024, 51(4): 353-358.
[11] ZHENG Cheng, SHI Jingwei, WEI Suhua, CHENG Jiaming. Dual Feature Adaptive Fusion Network Based on Dependency Type Pruning for Aspect-basedSentiment Analysis [J]. Computer Science, 2024, 51(3): 205-213.
[12] QIN Xianping, DING Zhaoxu, ZHONG Guoqiang, WANG Dong. Deep Learning-based Method for Mining Ocean Hot Spot News [J]. Computer Science, 2024, 51(11A): 231200005-10.
[13] CHEN Jing, ZHOU Gang, LI Shunhang, ZHENG Jiali, LU Jicang, HAO Yaohui. Review of Fake News Detection on Social Media [J]. Computer Science, 2024, 51(11): 1-14.
[14] LIU Feng, LIU Yaxuan, CHAI Xinyu, JI Haohan, ZHENG Zhixing. Computational Perception Technologies in Intelligent Education:Systematic Review [J]. Computer Science, 2024, 51(10): 10-16.
[15] FENG Jun, LI Kaixuan, GAO Zhizezhang, HUANG Li, SUN Xia. Survey of Research on Automated Grading Algorithms for Subjective Questions [J]. Computer Science, 2024, 51(10): 33-39.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!