计算机科学 ›› 2023, Vol. 50 ›› Issue (7): 66-71.doi: 10.11896/jsjkx.220900125

• 数据库&大数据&数据科学 • 上一篇    下一篇

基于变分持续贝叶斯元学习的推荐算法

朱文韬1,3, 刘威1,3, 梁上松1,3, 朱怀杰2,3, 印鉴2,3   

  1. 1 中山大学计算机学院 广州 511436
    2 中山大学人工智能学院 广东 珠海 519082
    3 广东省大数据分析与处理重点实验室 广州 510006
  • 收稿日期:2022-09-24 修回日期:2023-02-07 出版日期:2023-07-15 发布日期:2023-07-05
  • 通讯作者: 刘威(liuw259@mail.sysu.edu.cn)
  • 作者简介:(zhuwt8@mail2.sysu.edu.cn)
  • 基金资助:
    国家自然科学基金(U1911203,61902439,61902438,62002396);广东省基础与应用基础研究基金(2021A1515011902,2020A1515011251,2019A1515011159,2019A1515011704)

Variational Continuous Bayesian Meta-learning Based Algorithm for Recommendation

ZHU Wentao1,3, LIU Wei1,3, LIANG Shangsong1,3, ZHU Huaijie2,3, YIN Jian2,3   

  1. 1 School of Computer Science and Engineering,Sun Yat-sen University,Guangzhou 511436,China
    2 School of Artificial Intelligence,Sun Yat-sen University,Zhuhai,Guangdong 519082,China
    3 Guangdong Key Laboratory of Big Data Analysis and Processing,Guangzhou 510006,China
  • Received:2022-09-24 Revised:2023-02-07 Online:2023-07-15 Published:2023-07-05
  • About author:ZHU Wentao,born in 2000,postgra-duate.Her main research interests include recommendation system and meta-learning.LIU Wei,born in 1989,Ph.D,associate researcher,is a member of China Computer Federation.His main research interests include recommendation system and spatio-temporal data mining.
  • Supported by:
    National Natural Science Foundation of China(U1911203,61902439,61902438,62002396) and Guangdong Basic and Applied Basic Research Foundation(2021A1515011902,2020A1515011251,2019A1515011159,2019A1515011704).

摘要: 元学习方法近年被引入推荐系统以缓解冷启动问题。现有元学习算法只能提高算法处理一组静态分布的数据集(任务)的能力。当面对多个服从非平稳分布的数据集时,现有模型往往会出现负知识转移以及灾难性遗忘问题,导致算法推荐性能大幅下降。探索了基于变分持续贝叶斯元学习(Variational Continuous Bayesian Meta-Learning,VC-BML)的推荐算法。首先,算法假设元参数服从动态混合高斯模型,使其具有更大的参数空间,提高了模型适应不同任务的能力,缓解了负知识转移问题。然后,VC-BML的任务集群数量由中国餐馆过程(Chinese Restaurant Process,CRP)来灵活确定,使得模型在不同的混合分量中存储不同任务分布的知识,并在类似任务出现时调用这些知识,有助于缓解传统算法中的灾难性遗忘问题。为了估计模型参数的后验概率,算法采用了一种更稳健的结构化变分推理方法来近似后验值,以避免遗忘知识。最后,VC-BML在4个非平稳分布的数据集上的表现均优于基准算法。与基于点估计的基准算法相比,VC-BML提高了模型的稳健型,有助于缓解灾难性遗忘问题。

关键词: 推荐算法, 冷启动, 元学习, 动态混合高斯模型

Abstract: Meta-learning methods have been introduced into recommendation algorithms in recent years to alleviate the problem of cold start.The existing meta-learning algorithms can only improve the ability of the algorithm to deal with a set of statically distributed data sets(tasks).When faced with multiple data sets subject to non-stationary distribution,the existing models often have negative knowledge transfer and catastrophic forgetting problems,resulting in a significant decline in algorithm recommendation performance.This paper explores a recommendation algorithm based on variational continuous Bayesian Meta-learning(VC-BML).Firstly,the algorithm assumes that the meta parameters follow the dynamic mixed Gaussian model,which makes it have a larger parameter space,improves the ability of the model to adapt to different tasks,and alleviates the problem of negative knowledge transfer.Then,the number of task clusters in VC-BML is flexibly determined by Chinese restaurant process(CRP),which enables the model to store knowledge of different task distributions in different mixed components and invoke this know-ledge when similar tasks occur,helping to alleviate the catastrophic forgetting problem in traditional algorithms.To estimate the posterior probabilities of model parameters,a more robust structured variational reasoning method is used to approximate the posterior values to avoid forgetting knowledge.Finally,VC-BML outperforms the baselines on all four datasets with non-stationary distributions.Compared with point-based estimation methods,VC-BML improves the robustness of the model and helps alle-viate the catastrophic forgetting problem.

Key words: Recommendation algorithm, Cold-start problem, Meta-learning, Dynamic Gaussian mixture model

中图分类号: 

  • TP391
[1]MCNEE S M,RIEDL J,KONSTAN J A.Being accurate is not enough:how accuracy metrics have hurt recommender systems[C]//Extended Abstracts(CHI’06).2006:1097-1101.
[2]LI X,SHE J.Collaborative variational autoencoder for recommender systems[C]//KDD.2017:305-314.
[3]OORD A V D,DIELEMAN S,SCHRAUWEN B.Deep content-based music recommendation[C]//NIPS.2013:2643-2651.
[4]WEI J,HE J H,CHEN K,et al.Collaborative filtering and deeplearning based recommendation system for cold start items[J].Expert Systems with Applications,2017,69:29-39.
[5]LEE H,IM J,JANG S,et al.Melu:Meta-learned user preferenceestimator for cold-start recommendation[C]//KDD.2019:1073-1082.
[6]DENEVI G,STAMOS D,CILIBERTO C,et al.Online-within-online meta-learning[C]//NIPS.2019:1-11.
[7]VILALTA R,DRISSI Y.A perspective view and survey ofmeta-learning[J].Artificial Intelligence Review,2002,18(2):77-95.
[8]FINN C,ABBEEL P,LEVINE S.Model-agnostic meta-learning for fast adaptation of deep networks[C]//ICML.2017:1126-1135.
[9]LIANG S S,YILMAZ E,KANOULAS E.Collaboratively tra-cking interests for user clustering in streams of short texts[J].IEEE Transactions on Knowledge Data Engineering,2018,31(2):257-272.
[10]ZHANG Q,FANG J Y,MENG Z Q,et al.Varisational Conti-nual Bayesian Meta-Learning[C]//NIPS.2021:24556-24568.
[11]CHENG H T,KOC L,HARMSEN J,et al.Wide deep learning for recommender systems[C]//DLRS.2016:7-10.
[12]HE X N,LIAO L Z,ZHANG H W,et al.Neural collaborative filtering[C]//WWW.2017:173-182.
[13]BARKAN O,KOENIGSTEIN N.Item2vec:Neural item embedding for collaborative filtering[C]//IEEE MLSP.2016:1-6.
[14]ZHANG F Z,YUAN N J,LIAN D F,et al.Collaborative know-ledge base embedding for recommender systems[C]//KDD.2016:353-362.
[15]NAIR V,EHINTON G.Rectified linear units improve restricted boltzmann machines vinodnair[C]//ICML-10.2010:807-814.
[16]FINN C,RAJESWARAN A,KAKADE S,et al.Online meta-learning[EB/OL].(2019-02-22) [2019-07-03].https://doi.org/10.48550/arXiv.1902.08438.
[17]KALAI A,VEMPALA S.Efficient algorithms for online decision problems[J].Journal of Computer and System Sciences,2005,71(3):291-307.
[18]JERFEL G,GRANT E,GRIFFITHS T,et al.Reconciling meta-learning and continual learning with online mixtures of tasks[EB/OL].(2018-12-14) [2019-06-19].https://doi.org/10.48550/arXiv.1812.06080.
[19]YAP P,RITTER H,BARBER D.Bayesian online meta-lear-ning with Online Laplace Approximation[J].arXiv:2005.00146,2020.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!