基于循环时间卷积网络的序列流推荐算法

doi:10.11896/jsjkx.190500183

计算机科学 ›› 2020, Vol. 47 ›› Issue (3): 103-109.doi: 10.11896/jsjkx.190500183

• 数据库&大数据&数据科学 • 上一篇下一篇

基于循环时间卷积网络的序列流推荐算法

李太松^1,2,贺泽宇^1,2,王冰^1,2,颜永红^1,2,3,唐向红⁴

(中国科学院声学研究所语言声学与内容理解重点实验室北京100190)¹;
(中国科学院大学电子电气与通信工程学院北京100190)²;
(中国科学院新疆理化技术研究所新疆民族语音语言信息处理重点实验室乌鲁木齐 830011)³;
(贵州大学现代制造技术教育部重点实验室贵阳 550025)⁴

收稿日期:2019-05-31 出版日期:2020-03-15 发布日期:2020-03-30
通讯作者: 王冰(ariel.bingwang@outlook.com)
基金资助:
国家自然科学基金(11590770-4,11722437,61650202,U1536117,61671442,11674352,11504406,61601453);国家重点研发计划(2016YFB0801203,2016YFC0800503,2017YFB1002803);新疆维吾尔自治区重大科技专项(2016A03007-1);贵州省留学回国人员科技活动择优资助项目(2018.0002)

Session-based Recommendation Algorithm Based on Recurrent Temporal Convolutional Network

LI Tai-song^1,2,HE Ze-yu^1,2,WANG Bing^1,2,YAN Yong-hong^1,2,3,TANG Xiang-hong⁴

(Key Laboratory of Speech Acoustics and Content Understanding, Institute of Acoustics, Chinese Academy of Sciences, Beijing 100190, China)¹;
(School of Electronic, Electrical and Communication Engineering, University of Chinese Academy Sciences, Beijing 100190, China)²;
(Xinjiang Key Laboratory of Minority Speech and Language Information Processing, Xinjiang Technical Institute of Physics and Chemistry, Chinese Academy of Sciences, Urumchi 830011, China)³;
(Key Laboratory of Advanced Manufacturing Technology of the Ministry of Education, Guizhou University, Guiyang 550025, China)⁴

Received:2019-05-31 Online:2020-03-15 Published:2020-03-30
About author:LI Tai-song,born in 1990,doctorial student.His main research interests include recommendation system and data mining. WANG Bing,born in 1984,Ph.D,asso-ciate professor.Her main research interests include recommendation system and data mining.
Supported by:
This work was supported by the National Natural Science Foundation of China (11590770-4, 11722437, 61650202, U1536117, 61671442, 11674352, 11504406, 61601453), National Key Research and Development Program (2016YFB0801203, 2016YFC0800503, 2017YFB1002803), Key Science and Technology Project of the Xinjiang Uygur Autonomous Region (2016A03007-1) and Project of Guizhou High-level Study Abroad Talents Innovation and Entrepreneurship (2018.0002).

摘要/Abstract

摘要： 针对循环神经网络(Recurrent Neural Network,RNN)模型在序列流推荐中只能从宏观上捕捉序列的演变模式,忽略了物品(Item)间内部的微观联系,无法长程建模序列数据的变化规律的问题,提出了多维度序列建模算法循环时间卷积网络(Recurrent Temporal Convolutional Network,RTCN)。首先,将每个物品表示成定长向量,采用多层因果卷积和扩张卷积操作扩大感受野范围,建立序列元素间的长程依赖关系。利用残差连接网络提取不同层次的特征信息,解决反向传播中梯度衰减甚至消失的问题。综合设计时间卷积网络(Temporal Convolutional Network,TCN)提取序列流中前后物品间的局部特征,将物品信息映射到隐藏空间,得到细粒度的特征向量。为进一步建立元素间的宏观联系,将特征向量依次输入门限循环单元(Gated Recurrent Unit,GRU),迭代更新现有隐藏状态并预测下一时刻的输出。RTCN通过时间卷积网络,从输入序列流提取出长时间、多维度、细粒度的局部关联特征;经过门限循环网络,建模序列间的长距离依赖关系,捕捉序列元素的演变模式,并预测下一个出现的物品。利用网站、手机应用和音乐3个不同场景中的数据对模型进行了实验。实验结果显示,RTCN模型在召回率(Recall)和平均排序倒数(MRR)两个指标上比RNN模型高出6%~13%,比传统推荐算法高出9%~59%。通过对比不同的损失函数,模型在交叉熵损失函数下表现最优。此外,由于TCN中的卷积层具有多通道的结构,当数据维度丰富时,该模型对物品和用户的上下文信息具有很强的综合能力。

关键词: 深度学习, 时间卷积网络, 推荐系统, 序列流推荐, 循环神经网络

Abstract: Since the Recurrent Neural Network (RNN) generally models transition patterns,ignores the inner connection of items and can’t model the long-term evolving patterns of sequential data in session-based recommendations.A Recurrent Temporal Convolutional Network (RTCN) was proposed.Firstly,each item in the sequence is embedded as a vector,the multi-layer casual convolutions and dilated convolutions are applied so that the receptive field is improved and the long-term connections are established.A residual network is stacked to extract features from different layers.Therefore,the gradient vanishing or even disappearing in back propagation can be solved.With above operations,a well-designed Temporal Convolutional Network (TCN) is established.It extracts local features from sequence items,maps item information into latent space and generates fine-grained feature vectors as results.To further explore the connections between items in macroscopic way,the feature vectors are feed into Gated Recurrent Unit (GRU).After multiple iterations and updates to hidden states,the model can make a prediction of the next item.RTCN can extract long-time,multi-dimension,fine-grained local features from inputs by adapting temporal convolutional network.It also models the long-distance connections between items,captures the transition patterns and infers the next items by using GRU networks.The experimental results demonstrate that the RTCN model outperforms 6%～13% than RNN-based model and 9%～59% than other traditional recommendation methods under the metrics of Recall and Mean Reciprocal Rank (MRR).By comparing different definitions of loss,RTCN performs best under the cross entropy loss function.Meanwhile,due to the TCN multi-channel structure,the proposed model has a high potential capacity to embedding context features of items and users when the dataset information is rich.

Key words: Deep learning, Recommendation system, Recurrent neural network, Session-based recommendation, Temporal convolutional network

中图分类号:

TP183

李太松,贺泽宇,王冰,颜永红,唐向红. 基于循环时间卷积网络的序列流推荐算法[J]. 计算机科学, 2020, 47(3): 103-109. https://doi.org/10.11896/jsjkx.190500183

LI Tai-song,HE Ze-yu,WANG Bing,YAN Yong-hong,TANG Xiang-hong. Session-based Recommendation Algorithm Based on Recurrent Temporal Convolutional Network[J]. Computer Science, 2020, 47(3): 103-109. https://doi.org/10.11896/jsjkx.190500183

参考文献

[1]HIDASI B,KARATZOGLOU A,BALTRUNAS L,et al.Session-based recommendations with recurrent neural networks[J].arXiv:1511.06939,2015.
[2]RUMELHART D E,HINTON G E,WILLIAMS R J.Learning representations by back-propagating errors[J].Nature,1986,323(6088):399-421.
[3]KOREN Y,BELL R,VOLINSKY C.Matrix factorization tech- niques for recommender systems[J].Computer,2009,42(8):30-37.
[4]WEIMER M,KARATZOGLOU A.Cofi rank-maximum margin matrix factorization for collaborative ranking[C]∥Proceedings of the Twenty-First Annual Conference on Neural Information Processing Systems.2008:1593-1600.
[5]HIDASI B,TIKK D.Fast ALS-Based tensor factorization for context-aware recommendation from implicit feedback[C]∥Joint European Conference on Machine Learning and Knowledge Discovery in Databases.Berlin:Springer,2012:67-82.
[6]SARWAR B,KARYPIS G,KONSTAN J,et al.Item-based collaborative filtering recommendation algorithms[C]∥International Conference on World Wide Web.ACM,2001:285-295.
[7]KOREN Y.Factorization meets theneighborhood:a multifaceted collaborative filtering model[C]∥ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.ACM,2008:426-434.
[8]HIDASI B,QUADRANA M,TIKK D.Parallel recurrent neural network architectures for feature-rich session-based recommendations[C]∥ACM Conference on Recommender Systems.ACM,2016:241-248.
[9]BOGINA V,KUFLIK T.Incorporating dwell time in session- based recommendations with recurrent Neural networks [C]∥CEUR Workshop Proceedings.2017:57-59.
[10]QUADRANA M,KARATZOGLOU A,HIDASI B,et al.Personalizing session-based recommendations with hierarchical recurrent neural networks[C]∥Eleventh ACM Conference on Recommender Systems.ACM,2017:130-137.
[11]BAI S,KOLTER J Z,KOLTUN V.An empirical evaluation of generic convolutional and recurrent networks for sequence mo- deling [J].arXiv:1803.01271,2018.
[12]LIANG M,HU X.Recurrent convolutional neural network for object recognition[C]∥Computer Vision and Pattern Recognition.IEEE,2015:3367-3375.
[13]PINHEIRO P H O,COLLOBERT R.Recurrent convolutional neural networks for scene labeling[C]∥InternationalConfe-rence on International Conference on Machine Learning.2014:82-90.
[14]LÉCUN Y,BOTTOU L,BENGIO Y,et al.Gradient-based learning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[15]LONG J,SHELHAMER E,DARRELL T.Fully convolutional networks for semantic segmentation[C]∥IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer So-ciety,2015:3431-3440.
[16]HE K,ZHANG X,REN S,et al.Deep residual learning for ima- ge recognition[C]∥IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2016:770-778.
[17]SALIMANS T,KINGMA D P.Weight normalization:A simple reparameterization to accelerate training of deep neural networks[C]∥Advances in Neural Information Processing Systems.2016:901-909.
[18]CHUNG J,GULCEHRE C,CHO K H,et al.Empirical evaluation of gated recurrent neural networks on sequence modeling[J].arXiv:1412.3555,2014.
[19]BEN-SHIMON D,TSIKINOVSKY A,FRIEDMANN M,et al.Recsys challenge 2015 and the yoochoose dataset[C]∥RecSys’15:Proceedings of the 9th ACM Conference on Recommender Systems.New York:ACM,2015:357-358.
[20]CHO E,MYERS S A,LESKOVEC J.Friendship and mobility:user movement in locationbased social networks[C]∥Procee-dings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.ACM,2011:1082-1090.
[21]CELMA O.Music Recommendation and Discovery in the Long Tail[M].Springer,2010.

相关文章 15

[1]	饶志双, 贾真, 张凡, 李天瑞. 基于Key-Value关联记忆网络的知识图谱问答方法 Key-Value Relational Memory Networks for Question Answering over Knowledge Graph 计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[2]	汤凌韬, 王迪, 张鲁飞, 刘盛云. 基于安全多方计算和差分隐私的联邦学习方案 Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy 计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[3]	程章桃, 钟婷, 张晟铭, 周帆. 基于图学习的推荐系统研究综述 Survey of Recommender Systems Based on Graph Learning 计算机科学, 2022, 49(9): 1-13. https://doi.org/10.11896/jsjkx.210900072
[4]	王冠宇, 钟婷, 冯宇, 周帆. 基于矢量量化编码的协同过滤推荐方法 Collaborative Filtering Recommendation Method Based on Vector Quantization Coding 计算机科学, 2022, 49(9): 48-54. https://doi.org/10.11896/jsjkx.210700109
[5]	徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺. 时序知识图谱表示学习 Temporal Knowledge Graph Representation Learning 计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[6]	王剑, 彭雨琦, 赵宇斐, 杨健. 基于深度学习的社交网络舆情信息抽取方法综述 Survey of Social Network Public Opinion Information Extraction Based on Deep Learning 计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[7]	郝志荣, 陈龙, 黄嘉成. 面向文本分类的类别区分式通用对抗攻击方法 Class Discriminative Universal Adversarial Attack for Text Classification 计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[8]	姜梦函, 李邵梅, 郑洪浩, 张建朋. 基于改进位置编码的谣言检测模型 Rumor Detection Model Based on Improved Position Embedding 计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[9]	秦琪琦, 张月琴, 王润泽, 张泽华. 基于知识图谱的层次粒化推荐方法 Hierarchical Granulation Recommendation Method Based on Knowledge Graph 计算机科学, 2022, 49(8): 64-69. https://doi.org/10.11896/jsjkx.210600111
[10]	方义秋, 张震坤, 葛君伟. 基于自注意力机制和迁移学习的跨领域推荐算法 Cross-domain Recommendation Algorithm Based on Self-attention Mechanism and Transfer Learning 计算机科学, 2022, 49(8): 70-77. https://doi.org/10.11896/jsjkx.210600011
[11]	孙奇, 吉根林, 张杰. 基于非局部注意力生成对抗网络的视频异常事件检测方法 Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection 计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[12]	侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木. 中文预训练模型研究进展 Advances in Chinese Pre-training Models 计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[13]	周慧, 施皓晨, 屠要峰, 黄圣君. 基于主动采样的深度鲁棒神经网络学习 Robust Deep Neural Network Learning Based on Active Sampling 计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
[14]	苏丹宁, 曹桂涛, 王燕楠, 王宏, 任赫. 小样本雷达辐射源识别的深度学习方法综述 Survey of Deep Learning for Radar Emitter Identification Based on Small Sample 计算机科学, 2022, 49(7): 226-235. https://doi.org/10.11896/jsjkx.210600138
[15]	彭双, 伍江江, 陈浩, 杜春, 李军. 基于注意力神经网络的对地观测卫星星上自主任务规划方法 Satellite Onboard Observation Task Planning Based on Attention Neural Network 计算机科学, 2022, 49(7): 242-247. https://doi.org/10.11896/jsjkx.210500093

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

基于循环时间卷积网络的序列流推荐算法

Session-based Recommendation Algorithm Based on Recurrent Temporal Convolutional Network

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

Metrics

本文评价

推荐阅读 0