一种基于Meta-learning改进的特征交互算法

doi:10.11896/jsjkx.230100087

摘要/Abstract

摘要： 特征交互在推荐系统领域的广告点击率(Click-Through Rate,CTR)预测任务中至关重要,当前业界做的特征交互往往是基于内积、外积等矩阵变换,这些操作没有引入额外的信息,可以作为衡量两个向量相似性的手段,但作为特征交互的表示不一定是可靠的,许多特征交互无法有效提高点击率预测性能。首先从改善特征交互方式的角度入手引入额外的参数来学习一个映射,假设这个映射能够将两个向量的表征映射成交互的表征。学习映射的过程能够通过元学习(Meta-learning)来实现,故构建一个学习器以函数的方式表征特征交互。另外,不同的特征对不一定采取相同的方式交互,不能通过同一种交互方式得到所有特征对,因此设计一组元学习器(meta-learner)来学习映射函数,引入门控网络(GateNet)学习模型中元学习器的分布,那么不同的特征嵌入可以由一组元学习器得到表征。基于以上两点提出了一种融合多个元学习器并结合门控网络(Multiple meta-learners combined with GateNet,gate-MML)的特征交互算法,通过学习不同特征的联系和差异提高每个特征交互的质量。为了验证所提算法的性能,在xDeepFM模型上采用gate-MML做进一步的特征交互,采用2个真实广告点击率预测的数据集进行实验,并使用Logloss作为损失函数,AUC作为评价指标。实验结果表明与传统的CTR预测模型相比,改进算法提升了广告点击率预测任务的预测性能。

关键词: 特征交互, 广告点击率预测, 元学习, 门控网络, 推荐系统

Abstract: Feature interaction is crucial in the field of advertising click-through rate(CTR) prediction in recommendation systems.However,current industry practices for feature interaction often rely on matrix transformations such as inner and outer products,which do not introduce additional information and can only serve as a means of measuring the similarity between two vectors.Therefore,such methods may not reliably represent feature interaction and may not effectively improve the performance of CTR prediction.To address this issue,this paper first introduces additional parameters to learn a mapping from the perspective of improving the feature interaction,assuming that this mapping can map the representation of two vectors to the representation of interaction.The process of learning mapping can be achieved through meta-learning,which constructs a learner to represent feature interactions in a functional manner.Additionally,different features may not adopt the same interaction method,and it is impossible to obtain all feature pairs through a single interaction method.Therefore,a set of meta-lear-ners is designed to learn the mapping function,and a GateNet is introduced to learn the distribution of meta-learners in the model,so that a set of meta-learners can represent different feature embeddings.Based on these two points,a feature interaction algorithm is proposed that combines multiple meta-learners with GateNet(gate-MML),which improves the quality of each feature interaction by learning the connections and differences between different features.To verify the performance of the proposed algorithm,gate-MML is used for further feature interaction in the xDeepFM model,and experiments are conducted on two real advertising click-through rate prediction datasets using Logloss as the loss function and AUC as the evaluation metric.Experimental results show that compared with traditional CTR prediction models,the improved algorithm enhances the prediction performance of advertising click-through rate prediction tasks.

Key words: Feature interaction, Advertising click-through rate prediction, Meta-learning, GateNet, Recommender system

中图分类号:

TP181

白静, 耿新宇, 易流, 穆禹锟, 陈琴, 宋杰. 一种基于Meta-learning改进的特征交互算法[J]. 计算机科学, 2023, 50(11A): 230100087-8. https://doi.org/10.11896/jsjkx.230100087

BAI Jing, GENG Xinyu, YI Liu, MU Yukun, CHEN Qin, SONG Jie. Improved Feature Interaction Algorithm Based on Meta-learning[J]. Computer Science, 2023, 50(11A): 230100087-8. https://doi.org/10.11896/jsjkx.230100087

参考文献

[1]RICHARDSON M,DOMINOWSKA E,RAGNO R.Predicting Clicks:Estimating the Click-through Rate for New Ads[C]//Proceedings of the 16th International Conference on World Wide Web.Banff,Alberta,Canada:Association for Computing Machinery,2007:521-530.
[2]MCMAHAN H B,HOLT G,SCULLEY D,et al.Ad Click Prediction:a View from the Trenches[C]// Proceedings of the 19th ACM SIGKDD International Conference on KnowledgeDisco-very and Data Mining.2013.
[3]RENDLE S.Factorization Machines[C]//2010 IEEE International Conference on Data Mining.Sydney,NSW,Australia:IEEE,2010:995-1000.
[4]CHANG Y W,HSIEH C J,CHANG K W,et al.Training and testing low-degree polynomial data mappings via linear SVM[J].The Journal of Machine Learning Research,2010,8(11):1471-1490.
[5]BLONDEL M,FUJINO A,UEDA N,et al.Higher-order FactorizationMachines[C]//Advances in Neural Information Processing Systems.2016:3351-3359.
[6]HE X,PAN J,JIN O,et al.Practical Lessons from PredictingClicks on Ads at Facebook[C]//Proceedings of the Eighth International Workshop on Data Mining for Online Advertising.New York,NY,USA:Association for Computing Machinery,2014:1-9.
[7]ZHANG W,QIN J,GUO W,et al.Deep learning for click-through rate estimation[J].arXiv:2104.10584,2021.
[8]SHAN Y,HOENS T R,JIAO J,et al.Deep Crossing:Web-Scale Modeling without Manually Crafted Combinatorial Features[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.New York,NY,USA:Association for Computing Machinery,2016:255-262.
[9]CHENG H T,KOC L,HARMSEN J,et al.Wide & DeepLearning for Recommender Systems[C]//Proceedings of the 1st Workshop on Deep Learning for Recommender Systems.New York,USA:Association for Computing Machinery,2016:7-10.
[10]ZHANG W,DU T,WANG J.Deep Learning over Multi-ﬁeld Categorical Data[C]//Advances in Information Retrieval.Cham:Springer International Publishing,2016:45-57.
[11]QU Y,CAI H,REN K,et al.Product-Based Neural Networksfor User Response Prediction[C]//2016 IEEE 16th Interna-tional Conference on Data Mining(ICDM).Barcelona,Spain:IEEE,2016:1149-1154.
[12]LIU B,TANG R,CHEN Y,et al.Feature Generation by Convolutional Neural Network for Click-through Rate Prediction[C]//The World Wide Web Conference.2019:1119-1129.
[13]SONG W,SHI C,XIAO Z,et al.Autoint:Automatic Feature Interaction Learning via Self-attentive Neural Networks[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management.2019:1161-1170.
[14]LI Z,CHENG W,CHEN Y,et al.Interpretable Click-throughRate Predictionthrough Hierarchical Attention[C]//Procee-dings of the 13th International Conference on Web Search and Data Mining.2020:313-321.
[15]YANG Y,ZHAI P.Click-through rate prediction in online advertising:A literature review[J].Information Processing & Management,2022,59(2):102853.
[16]THRUN S,PRATT L.Learning to learn:Introduction andoverview[M]//Springer,Boston,MA,1998:3-17.
[17]WANG J X.Meta-learning in natural and artificial intelligence[J].Current Opinion in Behavioral Sciences,2021,38:90-95.
[18]HOSPEDALES T,ANTONIOU A,MICAELLI P,et al.Meta-learning in neural networks:A survey[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2021,44(9):5149-5169.
[19]FINN C,ABBEEL P,LEVINE S.Model-agnostic Meta-learning for Fast Adaptationof Deep Networks[C]//International Conference on Machine Learning.PMLR,2017:1126-1135.
[20]TAKIKAWA T,ACUNA D,JAMPANI V,et al.Gated-scnn:Gated Shape Cnns for Semantic Segmentation[C]//Proceedings of the IEEE/CVF International Conference on Computervision.2019:5229-5238.
[21]LI C,LI L,QI J.A Self-attentive Model with Gate Mechanism for Spoken Language Understanding[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.2018:3824-3833.
[22]YANG L,ZHONG J,ZHANG Y,et al.An improving faster-RCNN with multi-attention ResNet for small target detection in intelligent autonomous transport with 6G[J].IEEE Transactions on Intelligent Transportation Systems,2022,24(7):7717-7725.
[23]JUAN Y,ZHUANG Y,CHIN W S,et al.Field-aware Factorization Machines for CTR Prediction[C]//Proceedings of the 10th ACM Conference on Recommender Systems.New York,NY,USA:Association for Computing Machinery,2016:43-50.
[24]GUO H,TANG R,YE Y,et al.DeepFM:A Factorization-Machine based NeuralNetwork for CTR Prediction[C]//Procee-dings of the Twenty-Sixth International Joint Conference on Artiﬁcial Intelligence.Melbourne,Australia:AAAI Press,2017:1725-1731.
[25]WANG R,FU B,FU G,et al.Deep & Cross Network for Ad Click Predictions[C]//Proceedings of the 23rd ACM SIGKDD Conference on Knowledge Discovery and Data Mining.New York,NY,USA:Association for Computing Machinery,2017.
[26]LIAN J,ZHOU X,ZHANG F,et al.xdeepfm:Combining Explicit and Implicit Feature Interactions for Recommender Systems[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.2018:1754-1763.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed