Computer Science ›› 2022, Vol. 49 ›› Issue (7): 10-17.doi: 10.11896/jsjkx.210600009

• Database & Big Data & Data Science • Previous Articles     Next Articles

Click-Through Rate Prediction Model Based on Neural Architecture Search

SHUAI Jian-bo, WANG Jin-ce, HUANG Fei-hu, PENG Jian   

  1. College of Computer Science,Sichuan University,Chengdu 610065,China
  • Received:2021-06-02 Revised:2021-10-19 Online:2022-07-15 Published:2022-07-12
  • About author:SHUAI Jian-bo,born in 1996,postgra-duate,is a student member of China Computer Federation.His main research interests include recommendation algorithms and data mining.
    PENG Jian,born in 1970,Ph.D,professor,Ph.D supervisor,is a outstanding member of China Computer Federation.His main research interests include big data and wireless sensor network.
  • Supported by:
    National Key R&D Program of China(2017YFB0202403) and Key R&D Program of Sichuan Province,China(2020YFG0308,2019YFG0494,2020YFG0304,2017GZDZX0003).

Abstract: Click-through rate(CTR) prediction is an important task in the recommendation system.Its goal is to predict the pro-bability of a user clicking on an advertisement or item.Feature embedding and feature interacting are critical for prediction performance.Therefore,the ideas of many click-through rate prediction models are optimized based on these two aspects.However,most of the work only focus on one of the aspects,and almost all models do not distinguish features in feature interacting.The same embedding and interacting method are used when crossing the same feature with other features,which hinders the improvement of model performance.In order to solve this problem,the automatic super-field-aware feature embedding and interacting(Auto-SEI) model is proposed.Firstly,it assigns each sub-field to a super-field,and obtains the feature embedding according to the grouping,then selects appropriate interacting method for the feature pair to obtain the cross feature,and finally makes prediction.In Auto-SEI model,the division of sub-field and the selection of interacting methods are parameterized as an architecture search problem,and the neural architecture search(NAS) algorithm is used to compress the search space and make selections.A large number of experiments are conducted on three real large-scale data sets and the results show the excellent performance of the Auto-SEI model on the task of click-through rate prediction.

Key words: Click-Through rate prediction, Feature embedding, Feature interacting, Neural Architecture Search, Recommendation system

CLC Number: 

  • TP391
[1]GUO H,TANG R,YE Y,et al.DeepFM:a factorization-ma-chine based neural network for CTR prediction[C]//Procee-dings of the 26th International Joint Conference on Artificial Intelligence.2017:1725-1731.
[2]CHEN W,ZHAN L,CI Y,et al.FLEN:Leveraging Field for Scalable CTR Prediction[J].arXiv:1911.04690,2019.
[3]SONG W,SHI C,XIAO Z,et al.Autoint:Automatic feature interaction learning via self-attentive neural networks[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management.2019:1161-1170.
[4]WANG R,FU B,FU G,et al.Deep & cross network for ad click predictions [C]//Proceedings of the ADKDD'17.2017:1-7.
[5]LIAN J,ZHOU X,ZHANG F,et al.xdeepfm:Combining expli-cit and implicit feature interactions for recommender systems[C]//Proceedings of the 24th ACM SIGKDD International Confe-rence on Knowledge Discovery & Data Mining.2018:1754-1763.
[6]PAN J,XU J,RUIZ A L,et al.Field-weighted factorization machines for click-through rate prediction in display advertising[C]//Proceedings of the 2018 World Wide Web Conference.2018:1349-1357.
[7]HUANG F,QIAO S,PENG J,et al.STPR:A personalized next point-of-interest recommendation model with spatio-temporal effects based on purpose ranking[J].IEEE Transactions on Emerging Topics in Computing,2019,9(2):994-1005.
[8]RENDLE S.Factorization machines[C]//2010 IEEE Interna-tional Conference on Data Mining.IEEE,2010:995-1000.
[9]JUAN Y,ZHUANG Y,CHIN W S,et al.Field-aware factorization machines for CTR prediction[C]//Proceedings of the 10th ACM Conference on Recommender Systems.2016:43-50.
[10]WANG X,HE X,WANG M,et al.Neural graph collaborative filtering[C]//Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval.2019:165-174.
[11]QU Y,FANG B,ZHANG W,et al.Product-based neural networks for user response prediction over multi-field categorical data[J].ACM Transactions on Information Systems(TOIS),2018,37(1):1-35.
[12]YAO Q,CHEN X,KWOK J T,et al.Efficient neural interaction function search for collaborative filtering[C]//Proceedings of The Web Conference 2020.2020:1660-1670.
[13]LIU B,XUE N,GUO H,et al.AutoGroup:Automatic feature grouping for modelling explicit high-order feature interactions in CTR prediction[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval.2020:199-208.
[14]LIU B,ZHU C,LI G,et al.Autofis:Automatic feature interaction selection in factorization models for click-through rate prediction[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.2020:2636-2645.
[15]LUO Y,WANG M,ZHOU H,et al.Autocross:Automatic feature crossing for tabular data in real-world applications[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.2019:1936-1945.
[16]LIU W,TANG R,LI J,et al.Field-aware probabilistic embedding neural network for ctr prediction[C]//Proceedings of the 12th ACM Conference on Recommender Systems.2018:412-416.
[17]XIAO J,YE H,HE X,et al.Attentional factorization machines:Learning the weight of feature interactions via attention networks[J].arXiv:1708.04617,2017.
[18]VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[C]//Proceedings of the 31st International Confe-rence on Neural Information Processing Systems.2017:6000-6010.
[19]LIU B,TANG R,CHEN Y,et al.Feature generation by convolutional neural network for click-through rate prediction[C]//The World Wide Web Conference.2019:1119-1129.
[20]CHENG H T,KOC L,HARMSEN J,et al.Wide & deep lear-ning for recommender systems[C]//Proceedings of the 1st Workshop on Deep Learning for Recommender Systems.2016:7-10.
[21]HE X,CHUA T S.Neural factorization machines for sparsepredictive analytics[C]//Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval.2017:355-364.
[22]YANG Y,XU B,SHEN S,et al.Operation-aware Neural Networks for user response prediction[J].Neural Networks,2020,121:161-168.
[23]SONG Q,CHENG D,ZHOU H,et al.Towards automated neural interaction discovery for click-through rate prediction[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.2020:945-955.
[24]YAO Q,XU J,TU W W,et al.Efficient neural architecture search via proximal iterations[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:6664-6671.
[25]LIU H,SIMONYAN K,YANG Y.Darts:Differentiable architecture search[J].arXiv:1806.09055,2018.
[26]XIE S,ZHENG H,LIU C,et al.SNAS:stochastic neural architecture search[J].arXiv:1812.09926,2018.
[1] QIN Qi-qi, ZHANG Yue-qin, WANG Run-ze, ZHANG Ze-hua. Hierarchical Granulation Recommendation Method Based on Knowledge Graph [J]. Computer Science, 2022, 49(8): 64-69.
[2] FANG Yi-qiu, ZHANG Zhen-kun, GE Jun-wei. Cross-domain Recommendation Algorithm Based on Self-attention Mechanism and Transfer Learning [J]. Computer Science, 2022, 49(8): 70-77.
[3] QI Xiu-xiu, WANG Jia-hao, LI Wen-xiong, ZHOU Fan. Fusion Algorithm for Matrix Completion Prediction Based on Probabilistic Meta-learning [J]. Computer Science, 2022, 49(7): 18-24.
[4] CAI Xiao-juan, TAN Wen-an. Improved Collaborative Filtering Algorithm Combining Similarity and Trust [J]. Computer Science, 2022, 49(6A): 238-241.
[5] HE Yi-chen, MAO Yi-jun, XIE Xian-fen, GU Wan-rong. Matrix Transformation and Factorization Based on Graph Partitioning by Vertex Separator for Recommendation [J]. Computer Science, 2022, 49(6A): 272-279.
[6] HONG Zhi-li, LAI Jun, CAO Lei, CHEN Xi-liang, XU Zhi-xiong. Study on Intelligent Recommendation Method of Dueling Network Reinforcement Learning Based on Regret Exploration [J]. Computer Science, 2022, 49(6): 149-157.
[7] XIONG Zhong-min, SHU Gui-wen, GUO Huai-yu. Graph Neural Network Recommendation Model Integrating User Preferences [J]. Computer Science, 2022, 49(6): 165-171.
[8] YU Ai-xin, FENG Xiu-fang, SUN Jing-yu. Social Trust Recommendation Algorithm Combining Item Similarity [J]. Computer Science, 2022, 49(5): 144-151.
[9] CHEN Jin-peng, HU Ha-lei, ZHANG Fan, CAO Yuan, SUN Peng-fei. Convolutional Sequential Recommendation with Temporal Feature and User Preference [J]. Computer Science, 2022, 49(1): 115-120.
[10] ZHAN Wan-jiang, HONG Zhi-lin, FANG Lu-ping, WU Zhe-fu, LYU Yue-hua. Collaborative Filtering Recommendation Algorithm Based on Adversarial Learning [J]. Computer Science, 2021, 48(7): 172-177.
[11] YU Sheng, LI Bin, SUN Xiao-bing, BO Li-li, ZHOU Cheng. Approach for Knowledge-driven Similar Bug Report Recommendation [J]. Computer Science, 2021, 48(5): 91-98.
[12] XIAO Shi-tao, SHAO Ying-xia, SONG Wei-ping, CUI Bin. Hybrid Score Function for Collaborative Filtering Recommendation [J]. Computer Science, 2021, 48(3): 113-118.
[13] HAO Zhi-feng, LIAO Xiang-cai, WEN Wen, CAI Rui-chu. Collaborative Filtering Recommendation Algorithm Based on Multi-context Information [J]. Computer Science, 2021, 48(3): 168-173.
[14] XU Bing, YI Pei-yu, WANG Jin-ce, PENG Jian. High-order Collaborative Filtering Recommendation System Based on Knowledge Graph Embedding [J]. Computer Science, 2021, 48(11A): 244-250.
[15] LI Tai-song,HE Ze-yu,WANG Bing,YAN Yong-hong,TANG Xiang-hong. Session-based Recommendation Algorithm Based on Recurrent Temporal Convolutional Network [J]. Computer Science, 2020, 47(3): 103-109.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!