Computer Science ›› 2023, Vol. 50 ›› Issue (11A): 230100087-8.doi: 10.11896/jsjkx.230100087

• Big Data & Data Science • Previous Articles     Next Articles

Improved Feature Interaction Algorithm Based on Meta-learning

BAI Jing, GENG Xinyu, YI Liu, MU Yukun, CHEN Qin, SONG Jie   

  1. School of Computer Science,Southwest Petroleum University,Chengdu 610000,China
  • Published:2023-11-09
  • About author:BAI Jing,born in 1996,postgraduate.Her main research interests include machine learning,data mining and recommender system.
    GENG Xinyu,born in 1964,professor,master supervisor.His main research interests include data mining and artificial neural networks.
  • Supported by:
    Sichuan Science and Technology Program(2022NSFSC0555).

Abstract: Feature interaction is crucial in the field of advertising click-through rate(CTR) prediction in recommendation systems.However,current industry practices for feature interaction often rely on matrix transformations such as inner and outer products,which do not introduce additional information and can only serve as a means of measuring the similarity between two vectors.Therefore,such methods may not reliably represent feature interaction and may not effectively improve the performance of CTR prediction.To address this issue,this paper first introduces additional parameters to learn a mapping from the perspective of improving the feature interaction,assuming that this mapping can map the representation of two vectors to the representation of interaction.The process of learning mapping can be achieved through meta-learning,which constructs a learner to represent feature interactions in a functional manner.Additionally,different features may not adopt the same interaction method,and it is impossible to obtain all feature pairs through a single interaction method.Therefore,a set of meta-lear-ners is designed to learn the mapping function,and a GateNet is introduced to learn the distribution of meta-learners in the model,so that a set of meta-learners can represent different feature embeddings.Based on these two points,a feature interaction algorithm is proposed that combines multiple meta-learners with GateNet(gate-MML),which improves the quality of each feature interaction by learning the connections and differences between different features.To verify the performance of the proposed algorithm,gate-MML is used for further feature interaction in the xDeepFM model,and experiments are conducted on two real advertising click-through rate prediction datasets using Logloss as the loss function and AUC as the evaluation metric.Experimental results show that compared with traditional CTR prediction models,the improved algorithm enhances the prediction performance of advertising click-through rate prediction tasks.

Key words: Feature interaction, Advertising click-through rate prediction, Meta-learning, GateNet, Recommender system

CLC Number: 

  • TP181
[1]RICHARDSON M,DOMINOWSKA E,RAGNO R.Predicting Clicks:Estimating the Click-through Rate for New Ads[C]//Proceedings of the 16th International Conference on World Wide Web.Banff,Alberta,Canada:Association for Computing Machinery,2007:521-530.
[2]MCMAHAN H B,HOLT G,SCULLEY D,et al.Ad Click Prediction:a View from the Trenches[C]// Proceedings of the 19th ACM SIGKDD International Conference on KnowledgeDisco-very and Data Mining.2013.
[3]RENDLE S.Factorization Machines[C]//2010 IEEE International Conference on Data Mining.Sydney,NSW,Australia:IEEE,2010:995-1000.
[4]CHANG Y W,HSIEH C J,CHANG K W,et al.Training and testing low-degree polynomial data mappings via linear SVM[J].The Journal of Machine Learning Research,2010,8(11):1471-1490.
[5]BLONDEL M,FUJINO A,UEDA N,et al.Higher-order FactorizationMachines[C]//Advances in Neural Information Processing Systems.2016:3351-3359.
[6]HE X,PAN J,JIN O,et al.Practical Lessons from PredictingClicks on Ads at Facebook[C]//Proceedings of the Eighth International Workshop on Data Mining for Online Advertising.New York,NY,USA:Association for Computing Machinery,2014:1-9.
[7]ZHANG W,QIN J,GUO W,et al.Deep learning for click-through rate estimation[J].arXiv:2104.10584,2021.
[8]SHAN Y,HOENS T R,JIAO J,et al.Deep Crossing:Web-Scale Modeling without Manually Crafted Combinatorial Features[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.New York,NY,USA:Association for Computing Machinery,2016:255-262.
[9]CHENG H T,KOC L,HARMSEN J,et al.Wide & DeepLearning for Recommender Systems[C]//Proceedings of the 1st Workshop on Deep Learning for Recommender Systems.New York,USA:Association for Computing Machinery,2016:7-10.
[10]ZHANG W,DU T,WANG J.Deep Learning over Multi-field Categorical Data[C]//Advances in Information Retrieval.Cham:Springer International Publishing,2016:45-57.
[11]QU Y,CAI H,REN K,et al.Product-Based Neural Networksfor User Response Prediction[C]//2016 IEEE 16th Interna-tional Conference on Data Mining(ICDM).Barcelona,Spain:IEEE,2016:1149-1154.
[12]LIU B,TANG R,CHEN Y,et al.Feature Generation by Convolutional Neural Network for Click-through Rate Prediction[C]//The World Wide Web Conference.2019:1119-1129.
[13]SONG W,SHI C,XIAO Z,et al.Autoint:Automatic Feature Interaction Learning via Self-attentive Neural Networks[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management.2019:1161-1170.
[14]LI Z,CHENG W,CHEN Y,et al.Interpretable Click-throughRate Predictionthrough Hierarchical Attention[C]//Procee-dings of the 13th International Conference on Web Search and Data Mining.2020:313-321.
[15]YANG Y,ZHAI P.Click-through rate prediction in online advertising:A literature review[J].Information Processing & Management,2022,59(2):102853.
[16]THRUN S,PRATT L.Learning to learn:Introduction andoverview[M]//Springer,Boston,MA,1998:3-17.
[17]WANG J X.Meta-learning in natural and artificial intelligence[J].Current Opinion in Behavioral Sciences,2021,38:90-95.
[18]HOSPEDALES T,ANTONIOU A,MICAELLI P,et al.Meta-learning in neural networks:A survey[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2021,44(9):5149-5169.
[19]FINN C,ABBEEL P,LEVINE S.Model-agnostic Meta-learning for Fast Adaptationof Deep Networks[C]//International Conference on Machine Learning.PMLR,2017:1126-1135.
[20]TAKIKAWA T,ACUNA D,JAMPANI V,et al.Gated-scnn:Gated Shape Cnns for Semantic Segmentation[C]//Proceedings of the IEEE/CVF International Conference on Computervision.2019:5229-5238.
[21]LI C,LI L,QI J.A Self-attentive Model with Gate Mechanism for Spoken Language Understanding[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.2018:3824-3833.
[22]YANG L,ZHONG J,ZHANG Y,et al.An improving faster-RCNN with multi-attention ResNet for small target detection in intelligent autonomous transport with 6G[J].IEEE Transactions on Intelligent Transportation Systems,2022,24(7):7717-7725.
[23]JUAN Y,ZHUANG Y,CHIN W S,et al.Field-aware Factorization Machines for CTR Prediction[C]//Proceedings of the 10th ACM Conference on Recommender Systems.New York,NY,USA:Association for Computing Machinery,2016:43-50.
[24]GUO H,TANG R,YE Y,et al.DeepFM:A Factorization-Machine based NeuralNetwork for CTR Prediction[C]//Procee-dings of the Twenty-Sixth International Joint Conference on Artificial Intelligence.Melbourne,Australia:AAAI Press,2017:1725-1731.
[25]WANG R,FU B,FU G,et al.Deep & Cross Network for Ad Click Predictions[C]//Proceedings of the 23rd ACM SIGKDD Conference on Knowledge Discovery and Data Mining.New York,NY,USA:Association for Computing Machinery,2017.
[26]LIAN J,ZHOU X,ZHANG F,et al.xdeepfm:Combining Explicit and Implicit Feature Interactions for Recommender Systems[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.2018:1754-1763.
[1] WANG Huaiqin, LUO Jian, WANG Haiyan. Feature Weight Perception-based Prediction of Virtual Network Function Resource Demands [J]. Computer Science, 2023, 50(9): 331-336.
[2] HUANG Lu, NI Lyu, JIN Cheqing. Rectifying Dual Bias for Recommendation [J]. Computer Science, 2023, 50(9): 152-159.
[3] JIN Tiancheng, DOU Liang, ZHANG Wei, XIAO Chunyun, LIU Feng, ZHOU Aimin. OJ Exercise Recommendation Model Based on Deep Reinforcement Learning and Program Analysis [J]. Computer Science, 2023, 50(8): 58-67.
[4] WANG Jiahao, ZHONG Xin, LI Wenxiong, ZHAO Dexin. Human Activity Recognition with Meta-learning and Attention [J]. Computer Science, 2023, 50(8): 193-201.
[5] ZHU Wentao, LIU Wei, LIANG Shangsong, ZHU Huaijie, YIN Jian. Variational Continuous Bayesian Meta-learning Based Algorithm for Recommendation [J]. Computer Science, 2023, 50(7): 66-71.
[6] YANG Bin, LIANG Jing, ZHOU Jiawei, ZHAO Mengci. Study on Interpretable Click-Through Rate Prediction Based on Attention Mechanism [J]. Computer Science, 2023, 50(5): 12-20.
[7] SHANG Di, LYU Yanfeng, QIAO Hong. Incremental Object Detection Inspired by Memory Mechanisms in Brain [J]. Computer Science, 2023, 50(2): 267-274.
[8] ZHU Jun, HAN Lixin, ZONG Ping, XU Yiqing, XIA Ji’an, TANG Ming. Natural Noise Filtering Algorithm for Point-of-Interest Recommender Systems [J]. Computer Science, 2023, 50(11): 132-142.
[9] HAO Jingyu, WEN Jingxuan, LIU Huafeng, JING Liping, YU Jian. Deep Disentangled Collaborative Filtering with Graph Global Information [J]. Computer Science, 2023, 50(1): 41-51.
[10] CHENG Zhang-tao, ZHONG Ting, ZHANG Sheng-ming, ZHOU Fan. Survey of Recommender Systems Based on Graph Learning [J]. Computer Science, 2022, 49(9): 1-13.
[11] WANG Guan-yu, ZHONG Ting, FENG Yu, ZHOU Fan. Collaborative Filtering Recommendation Method Based on Vector Quantization Coding [J]. Computer Science, 2022, 49(9): 48-54.
[12] QI Xiu-xiu, WANG Jia-hao, LI Wen-xiong, ZHOU Fan. Fusion Algorithm for Matrix Completion Prediction Based on Probabilistic Meta-learning [J]. Computer Science, 2022, 49(7): 18-24.
[13] GUO Liang, YANG Xing-yao, YU Jiong, HAN Chen, HUANG Zhong-hao. Hybrid Recommender System Based on Attention Mechanisms and Gating Network [J]. Computer Science, 2022, 49(6): 158-164.
[14] CHEN Zhuang, ZOU Hai-tao, ZHENG Shang, YU Hua-long, GAO Shang. Diversity Recommendation Algorithm Based on User Coverage and Rating Differences [J]. Computer Science, 2022, 49(5): 159-164.
[15] ZHOU Ying, CHANG Ming-xin, YE Hong, ZHANG Yan. Super Resolution Reconstruction Method of Solar Panel Defect Images Based on Meta-transfer [J]. Computer Science, 2022, 49(3): 185-191.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!