计算机科学 ›› 2022, Vol. 49 ›› Issue (7): 10-17.doi: 10.11896/jsjkx.210600009

• 数据库&大数据&数据科学* 上一篇    下一篇

基于神经架构搜索的点击率预测模型

帅剑波, 王金策, 黄飞虎, 彭舰   

  1. 四川大学计算机学院 成都610065
  • 收稿日期:2021-06-02 修回日期:2021-10-19 出版日期:2022-07-15 发布日期:2022-07-12
  • 通讯作者: 彭舰(jianpeng@scu.edu.cn)
  • 作者简介:(122569217@qq.com)
  • 基金资助:
    国家重点研发计划(2017YFB0202403);四川省重点研发计划(2020YFG0308,2019YFG0494,2020YFG0304,2017GZDZX0003)

Click-Through Rate Prediction Model Based on Neural Architecture Search

SHUAI Jian-bo, WANG Jin-ce, HUANG Fei-hu, PENG Jian   

  1. College of Computer Science,Sichuan University,Chengdu 610065,China
  • Received:2021-06-02 Revised:2021-10-19 Online:2022-07-15 Published:2022-07-12
  • About author:SHUAI Jian-bo,born in 1996,postgra-duate,is a student member of China Computer Federation.His main research interests include recommendation algorithms and data mining.
    PENG Jian,born in 1970,Ph.D,professor,Ph.D supervisor,is a outstanding member of China Computer Federation.His main research interests include big data and wireless sensor network.
  • Supported by:
    National Key R&D Program of China(2017YFB0202403) and Key R&D Program of Sichuan Province,China(2020YFG0308,2019YFG0494,2020YFG0304,2017GZDZX0003).

摘要: 点击率(Click-Through Rate,CTR)预测是推荐系统中一项重要的任务,其目标是预测用户点击一个广告或者商品的概率。特征嵌入和特征组合是影响预测性能的关键,因此很多点击率预测模型的思路也是针对这两个方面进行优化。但目前大部分工作仅关注其中一个方面,并且几乎所有的模型在进行特征组合时都没有对特征进行区分,同一个特征与其他特征组合时都使用相同的嵌入和组合方法,阻碍了模型性能的提升。为解决该问题,提出了Auto-SEI(Automatic Super-field-aware Feature Embedding and Interacting)模型。该模型先将每个特征子域分配给一个特征超域,再根据分组得到特征的嵌入,然后为特征对选择合适的组合方法获取组合特征,最后进行预测。Auto-SEI模型中,特征子域的划分和组合方法的选择被参数化为架构搜索问题,利用神经架构搜索(Neural Architecture Search,NAS)算法压缩搜索空间并进行选择。在3个真实的大规模数据集上进行了大量实验,结果表明Auto-SEI 模型在点击率预测任务上具有优秀的性能。

关键词: 点击率预测, 神经架构搜索, 特征嵌入, 特征组合, 推荐系统

Abstract: Click-through rate(CTR) prediction is an important task in the recommendation system.Its goal is to predict the pro-bability of a user clicking on an advertisement or item.Feature embedding and feature interacting are critical for prediction performance.Therefore,the ideas of many click-through rate prediction models are optimized based on these two aspects.However,most of the work only focus on one of the aspects,and almost all models do not distinguish features in feature interacting.The same embedding and interacting method are used when crossing the same feature with other features,which hinders the improvement of model performance.In order to solve this problem,the automatic super-field-aware feature embedding and interacting(Auto-SEI) model is proposed.Firstly,it assigns each sub-field to a super-field,and obtains the feature embedding according to the grouping,then selects appropriate interacting method for the feature pair to obtain the cross feature,and finally makes prediction.In Auto-SEI model,the division of sub-field and the selection of interacting methods are parameterized as an architecture search problem,and the neural architecture search(NAS) algorithm is used to compress the search space and make selections.A large number of experiments are conducted on three real large-scale data sets and the results show the excellent performance of the Auto-SEI model on the task of click-through rate prediction.

Key words: Click-Through rate prediction, Feature embedding, Feature interacting, Neural Architecture Search, Recommendation system

中图分类号: 

  • TP391
[1]GUO H,TANG R,YE Y,et al.DeepFM:a factorization-ma-chine based neural network for CTR prediction[C]//Procee-dings of the 26th International Joint Conference on Artificial Intelligence.2017:1725-1731.
[2]CHEN W,ZHAN L,CI Y,et al.FLEN:Leveraging Field for Scalable CTR Prediction[J].arXiv:1911.04690,2019.
[3]SONG W,SHI C,XIAO Z,et al.Autoint:Automatic feature interaction learning via self-attentive neural networks[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management.2019:1161-1170.
[4]WANG R,FU B,FU G,et al.Deep & cross network for ad click predictions [C]//Proceedings of the ADKDD'17.2017:1-7.
[5]LIAN J,ZHOU X,ZHANG F,et al.xdeepfm:Combining expli-cit and implicit feature interactions for recommender systems[C]//Proceedings of the 24th ACM SIGKDD International Confe-rence on Knowledge Discovery & Data Mining.2018:1754-1763.
[6]PAN J,XU J,RUIZ A L,et al.Field-weighted factorization machines for click-through rate prediction in display advertising[C]//Proceedings of the 2018 World Wide Web Conference.2018:1349-1357.
[7]HUANG F,QIAO S,PENG J,et al.STPR:A personalized next point-of-interest recommendation model with spatio-temporal effects based on purpose ranking[J].IEEE Transactions on Emerging Topics in Computing,2019,9(2):994-1005.
[8]RENDLE S.Factorization machines[C]//2010 IEEE Interna-tional Conference on Data Mining.IEEE,2010:995-1000.
[9]JUAN Y,ZHUANG Y,CHIN W S,et al.Field-aware factorization machines for CTR prediction[C]//Proceedings of the 10th ACM Conference on Recommender Systems.2016:43-50.
[10]WANG X,HE X,WANG M,et al.Neural graph collaborative filtering[C]//Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval.2019:165-174.
[11]QU Y,FANG B,ZHANG W,et al.Product-based neural networks for user response prediction over multi-field categorical data[J].ACM Transactions on Information Systems(TOIS),2018,37(1):1-35.
[12]YAO Q,CHEN X,KWOK J T,et al.Efficient neural interaction function search for collaborative filtering[C]//Proceedings of The Web Conference 2020.2020:1660-1670.
[13]LIU B,XUE N,GUO H,et al.AutoGroup:Automatic feature grouping for modelling explicit high-order feature interactions in CTR prediction[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval.2020:199-208.
[14]LIU B,ZHU C,LI G,et al.Autofis:Automatic feature interaction selection in factorization models for click-through rate prediction[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.2020:2636-2645.
[15]LUO Y,WANG M,ZHOU H,et al.Autocross:Automatic feature crossing for tabular data in real-world applications[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.2019:1936-1945.
[16]LIU W,TANG R,LI J,et al.Field-aware probabilistic embedding neural network for ctr prediction[C]//Proceedings of the 12th ACM Conference on Recommender Systems.2018:412-416.
[17]XIAO J,YE H,HE X,et al.Attentional factorization machines:Learning the weight of feature interactions via attention networks[J].arXiv:1708.04617,2017.
[18]VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[C]//Proceedings of the 31st International Confe-rence on Neural Information Processing Systems.2017:6000-6010.
[19]LIU B,TANG R,CHEN Y,et al.Feature generation by convolutional neural network for click-through rate prediction[C]//The World Wide Web Conference.2019:1119-1129.
[20]CHENG H T,KOC L,HARMSEN J,et al.Wide & deep lear-ning for recommender systems[C]//Proceedings of the 1st Workshop on Deep Learning for Recommender Systems.2016:7-10.
[21]HE X,CHUA T S.Neural factorization machines for sparsepredictive analytics[C]//Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval.2017:355-364.
[22]YANG Y,XU B,SHEN S,et al.Operation-aware Neural Networks for user response prediction[J].Neural Networks,2020,121:161-168.
[23]SONG Q,CHENG D,ZHOU H,et al.Towards automated neural interaction discovery for click-through rate prediction[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.2020:945-955.
[24]YAO Q,XU J,TU W W,et al.Efficient neural architecture search via proximal iterations[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:6664-6671.
[25]LIU H,SIMONYAN K,YANG Y.Darts:Differentiable architecture search[J].arXiv:1806.09055,2018.
[26]XIE S,ZHENG H,LIU C,et al.SNAS:stochastic neural architecture search[J].arXiv:1812.09926,2018.
[1] 程章桃, 钟婷, 张晟铭, 周帆.
基于图学习的推荐系统研究综述
Survey of Recommender Systems Based on Graph Learning
计算机科学, 2022, 49(9): 1-13. https://doi.org/10.11896/jsjkx.210900072
[2] 王冠宇, 钟婷, 冯宇, 周帆.
基于矢量量化编码的协同过滤推荐方法
Collaborative Filtering Recommendation Method Based on Vector Quantization Coding
计算机科学, 2022, 49(9): 48-54. https://doi.org/10.11896/jsjkx.210700109
[3] 秦琪琦, 张月琴, 王润泽, 张泽华.
基于知识图谱的层次粒化推荐方法
Hierarchical Granulation Recommendation Method Based on Knowledge Graph
计算机科学, 2022, 49(8): 64-69. https://doi.org/10.11896/jsjkx.210600111
[4] 方义秋, 张震坤, 葛君伟.
基于自注意力机制和迁移学习的跨领域推荐算法
Cross-domain Recommendation Algorithm Based on Self-attention Mechanism and Transfer Learning
计算机科学, 2022, 49(8): 70-77. https://doi.org/10.11896/jsjkx.210600011
[5] 齐秀秀, 王佳昊, 李文雄, 周帆.
基于概率元学习的矩阵补全预测融合算法
Fusion Algorithm for Matrix Completion Prediction Based on Probabilistic Meta-learning
计算机科学, 2022, 49(7): 18-24. https://doi.org/10.11896/jsjkx.210600126
[6] 蔡晓娟, 谭文安.
一种改进的融合相似度和信任度的协同过滤算法
Improved Collaborative Filtering Algorithm Combining Similarity and Trust
计算机科学, 2022, 49(6A): 238-241. https://doi.org/10.11896/jsjkx.210400088
[7] 何亦琛, 毛宜军, 谢贤芬, 古万荣.
基于点割集图分割的矩阵变换与分解的推荐算法
Matrix Transformation and Factorization Based on Graph Partitioning by Vertex Separator for Recommendation
计算机科学, 2022, 49(6A): 272-279. https://doi.org/10.11896/jsjkx.210600159
[8] 洪志理, 赖俊, 曹雷, 陈希亮, 徐志雄.
基于遗憾探索的竞争网络强化学习智能推荐方法研究
Study on Intelligent Recommendation Method of Dueling Network Reinforcement Learning Based on Regret Exploration
计算机科学, 2022, 49(6): 149-157. https://doi.org/10.11896/jsjkx.210600226
[9] 郭亮, 杨兴耀, 于炯, 韩晨, 黄仲浩.
基于注意力机制和门控网络相结合的混合推荐系统
Hybrid Recommender System Based on Attention Mechanisms and Gating Network
计算机科学, 2022, 49(6): 158-164. https://doi.org/10.11896/jsjkx.210500013
[10] 熊中敏, 舒贵文, 郭怀宇.
融合用户偏好的图神经网络推荐模型
Graph Neural Network Recommendation Model Integrating User Preferences
计算机科学, 2022, 49(6): 165-171. https://doi.org/10.11896/jsjkx.210400276
[11] 余皑欣, 冯秀芳, 孙静宇.
结合物品相似性的社交信任推荐算法
Social Trust Recommendation Algorithm Combining Item Similarity
计算机科学, 2022, 49(5): 144-151. https://doi.org/10.11896/jsjkx.210300217
[12] 陈壮, 邹海涛, 郑尚, 于化龙, 高尚.
基于用户覆盖及评分差异的多样性推荐算法
Diversity Recommendation Algorithm Based on User Coverage and Rating Differences
计算机科学, 2022, 49(5): 159-164. https://doi.org/10.11896/jsjkx.210300263
[13] 陈晋鹏, 胡哈蕾, 张帆, 曹源, 孙鹏飞.
融合时间特性和用户偏好的卷积序列化推荐
Convolutional Sequential Recommendation with Temporal Feature and User Preference
计算机科学, 2022, 49(1): 115-120. https://doi.org/10.11896/jsjkx.201200192
[14] 詹皖江, 洪植林, 方路平, 吴哲夫, 吕跃华.
基于对抗性学习的协同过滤推荐算法
Collaborative Filtering Recommendation Algorithm Based on Adversarial Learning
计算机科学, 2021, 48(7): 172-177. https://doi.org/10.11896/jsjkx.200600077
[15] 余笙, 李斌, 孙小兵, 薄莉莉, 周澄.
知识驱动的相似缺陷报告推荐方法
Approach for Knowledge-driven Similar Bug Report Recommendation
计算机科学, 2021, 48(5): 91-98. https://doi.org/10.11896/jsjkx.200600159
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!