基于特征嵌入门控和多项式特征交叉网络的点击率预测模型

doi:10.11896/jsjkx.240900092

计算机科学 ›› 2025, Vol. 52 ›› Issue (6A): 240900092-6.doi: 10.11896/jsjkx.240900092

基于特征嵌入门控和多项式特征交叉网络的点击率预测模型

栾方军¹, 张凤强², 袁帅³

1 沈阳建筑大学计算机科学与工程学院沈阳 110168
2 辽宁省城市建设大数据管理与分析重点实验室沈阳 110168
3 国家特种计算机工程技术研究中心沈阳分中心沈阳 110168

出版日期:2025-06-16 发布日期:2025-06-12
通讯作者: 袁帅(reidyuan@163.com)
作者简介:(luanfangjun@sjzu.edu.cn)
基金资助:
国家自然科学基金项目(62073227);辽宁省应用基础研究计划项目(2023JH2/101300212)

Click-through Rate Prediction Model Based on Feature Embedding Gating and PolynomialFeature Crossover Networks

LUAN Fangjun¹, ZHANG Fengqiang², YUAN Shuai³

1 School of Computer Science and Engineering,Shenyang Jianzhu University,Shenyang 110168,China
2 Liaoning Province Big Data Management and Analysis Laboratory of Urban Construction,Shenyang 110168,China
3 Shenyang Branch of National Special Computer Engineering Technology Research Center,Shenyang 110168,China

Online:2025-06-16 Published:2025-06-12
About author:LUAN Fangjun,born in 1971,Ph.D,professor,Ph.D supervisor,is a senior member of CCF(No.15582S).His main research interests include pattern re-cognition and intelligent building.
YUAN Shuai,born in 1978,Ph.D,professor,Ph.D supervisor.His main research interests include deep learning,image recognition and robot control.
Supported by:
National Natural Science Foundation of China(62073227) and Liaoning Province Applied Basic Research Program Project(2023JH2/101300212).

摘要/Abstract

摘要： 点击率预测在推荐系统和在线广告中发挥着至关重要的作用,而特征嵌入和特征交互是影响预测准确性的关键因素。但许多现有模型主要集中于设计特征交互结构,并且它们通常采用简单的计算方法,如哈达玛积、内积、单一的向量级或位级特征交互或者结合多层感知机进行隐式特征交互,这些方法在处理复杂特征交互时可能存在局限性。针对以上不足,提出了基于特征嵌入门控和多项式特征交叉网络的点击率预测模型。首先,为了实现更有效的特征交互,提出了多项式特征交叉网络,网络通过结合哈达玛积和内积实现特征交叉,以递归的形式实现显式高阶特征交叉。接着,通过融合两个并行的多项式特征交叉网络进行向量级和位级的特征交叉,实现特征的细粒度交互。最后,为了动态学习特征嵌入的重要性,增加特征交互网络输入的差异性,提出了特征嵌入门控,门控可以从向量级和位级学习特征的权重,从而使交互网络更有针对性地捕捉不同的特征交互信息。在4个开放基准数据集上评估了模型性能,模型在Criteo数据集上AUC和Logloss分别达到了0.814 9和0.437 2;在Avazu数据集上AUC和Logloss分别达到了0.766 3和0.366 1;在Movielens数据集上AUC和Logloss分别达到了0.971 6和0.198 4;在Frappe数据集上AUC和Logloss分别达到了0.985 8和0.138 7。实验结果表明,所提模型在点击率预测中表现出更好的性能,有效提升了预测准确性。

Abstract: Click-through rate prediction plays a crucial role in recommender systems and online advertisements,and feature embedding and feature interactions are key factors affecting prediction accuracy.However,many existing models mainly focus on designing feature interaction structures,and they usually use simple computational methods such as Hadamard product,inner pro-duct,single vector-level or bit-level feature interaction or combining multilayer perceptron for implicit feature interaction,which may have limitations in dealing with complex feature interactions.Tomake up for the above shortcomings,a click-through rate prediction model based on feature embedding gating and polynomial feature crossover networks is proposed.First,in order to achieve more effective feature interactions,polynomial featurecrossover network is proposed,where the network realizes feature crossover by combining Hadamard product and inner product to achieve explicit higher-order feature crossover in a recursive form.Then,fine-grained feature interaction is achieved by fusing two parallel polynomial feature crossover networks for vector-level and bit-level feature crossover.Finally,in order to dynamically learn the importance of feature embeddings and increase the variability of the inputs to the feature interaction network,feature embedding gating is proposed,which learns the weights of the features from the vector level and the bit level so that the interaction network can be more targeted to capture different feature interaction information.The model performance is evaluated on four open benchmark datasets,and the model achieves AUC and Logloss of 0.814 9 and 0.437 2 respectively,on the Criteo dataset;0.766 3 and 0.366 1 on the Avazu dataset;0.971 6 and 0.198 4 on the Movielens dataset;and 0.985 8 and 0.138 7 on the Frappe dataset.The experimental results show that the proposedmodel exhibits better performance in click-through-rate prediction,and effectively improves the prediction accuracy.

Key words: Recommendation system, Click-through rate prediction, Feature interaction, Feature embedding gating, Fine-grained interaction

中图分类号:

TP391

栾方军, 张凤强, 袁帅. 基于特征嵌入门控和多项式特征交叉网络的点击率预测模型[J]. 计算机科学, 2025, 52(6A): 240900092-6. https://doi.org/10.11896/jsjkx.240900092

LUAN Fangjun, ZHANG Fengqiang, YUAN Shuai. Click-through Rate Prediction Model Based on Feature Embedding Gating and PolynomialFeature Crossover Networks[J]. Computer Science, 2025, 52(6A): 240900092-6. https://doi.org/10.11896/jsjkx.240900092

参考文献

[1]CHENG H T,KOC L,HARMSEN J,et al.Wide & deep lear-ning for recommender systems[C]//Proceedings of the 1st Workshop on Deep Learning for Recommender Systems.2016:7-10.
[2]HUANG T,ZHANG Z,ZHANG J.FiBiNET:combining featureimportance and bilinear feature interaction for click-through rate prediction[C]//Proceedings of the 13th ACM Conference on Cecommender Systems.2019:169-177.
[3]CHEN B,WANG Y,LIU Z,et al.Enhancing Explicit and Implicit Feature Interactions via Information Sharing for Parallel Deep CTR Models[C]//Proceedings of the 30th ACM International Conference on Information and Knowledge Management.2021:3757-3766.
[4]RICHARDSON M,DOMINOWSKA E,RAGNO R.Predicting clicks:estimating the click-through rate for new ads[C]//Proceedings of the 16th International Conference on World Wide Web.2007:521-530.
[5]RENDLE S.Factorization machines[C]//2010 IEEE International Conference on Data Mining.IEEE,2010:995-1000.
[6]GUO H,TANG R,YE Y,et al.DeepFM:a factorization-machine based neural network for CTR prediction[J].arXiv:1703.04247,2017.
[7]LIAN J,ZHOU X,ZHANG F,et al.xdeepfm:Combining explicit and implicit feature interactions for recommender systems[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data NMining.2018:1754-1763.
[8]WANG R,SHIVANNA R,CHENG D,et al.Dcn v2:Improved deep & cross network and practical lessons for web-scale lear-ning to rank systems[C]//Proceedings of the Web Conference 2021.2021:1785-1797.
[9]WANG R,FU B,FU G,et al.Deep & cross network for ad click predictions[C]//Proceedings of the ADKDD’17.2017:1-7.
[10]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[11]HU J,SHEN L,SUN G.Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:7132-7141.
[12]SONG W,SHI C,XIAO Z,et al.Autoint:Automatic feature interaction learning via self-attentive neural networks[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management.2019:1161-1170.
[13]YU Y,WANG Z,YUAN B.An Input-aware Factorization Machine for Sparse Prediction[C]//IJCAI.2019:1466-1472.
[14]LU W,YU Y,CHANG Y,et al.A dual input-aware factorization machine for CTR prediction[C]//Proceedings of the Twenty-ninth International Conference on International Joint Confe-rences on Artificial Intelligence.2021:3139-3145.
[15]WANG F,GU H,LI D,et al.MCRF:Enhancing CTR Prediction Models via Multi-channel Feature Refinement Framework[C]//International Conference on Database Systems for Advanced Applications.2022:359-374.
[16]WANG H,LI N.A Click-Through Rate Prediction MethodBased on Cross-Importance of Multi-Order Features[J].arXiv:2405.08852,2021.
[17]CHENG W,SHEN Y,HUANG L.Adaptive factorization net-work:Learning adaptive-order feature interactions[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:3609-3616.
[18]WANG F,WANG Y,LI D,et al.Enhancing CTR predictionwith context-aware feature representation learning[C]//Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval.2022:343-352.
[19]YANG B,LIANG J,ZHOU J W,et al.Research on an interpretable click-through rate prediction model based on attention mechanism[J].Computer Science,2023,50(5):12-20.
[20]ZHAO W T,XUE S L,LIU T T.Recommendation for reducing irrelevant neighbors by combining item attribute collaborative signals[J].Computer Engineering and Applications,2024,60(7):101-107.
[21]SUN Y,PAN J,ZHANG A,et al.FM2:Field-matrixed factori-zation machines for recommender systems[C]//Proceedings of the Web Conference 2021.2021:2828-2837.
[22]PAN J,XU J,RUIZ AL,et al.Field-weighted factorization machines for click-through rate prediction in display advertising[C]//Proceedings of the 2018 World Wide Web Conference.2018:1349-1357.
[23]WANG Z,SHE Q,ZHANG J.Masknet:Introducing feature-wise multiplication to CTR ranking models by instance-guided mask[J].arXiv:2102.07619,2021.
[24]WANG F,GU H,LI D,et al.Towards deeper,lighter and interpretable cross network for ctr prediction[C]//Proceedings of the 32nd ACM International Conference on Information and Knowledge Management.2023:2523-2533.
[25]ZHU J,DAI Q,SU L,et al.Bars:Towards open benchmarking for recommender systems[C]//Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval.2022:2912-2923.
[26]ZHU J,LIU J,YANG S,et al.Open benchmarking for click-through rate prediction[C]//Proceedings of the 30th ACM International Conference on Information & Knowledge Management.2021:2759-2769.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

基于特征嵌入门控和多项式特征交叉网络的点击率预测模型

Click-through Rate Prediction Model Based on Feature Embedding Gating and PolynomialFeature Crossover Networks

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 0

Metrics

本文评价

推荐阅读 0