基于循环时间卷积网络的序列流推荐算法

doi:10.11896/jsjkx.190500183

计算机科学 ›› 2020, Vol. 47 ›› Issue (3): 103-109.doi: 10.11896/jsjkx.190500183

• 数据库&大数据&数据科学 • 上一篇下一篇

基于循环时间卷积网络的序列流推荐算法

李太松^1,2,贺泽宇^1,2,王冰^1,2,颜永红^1,2,3,唐向红⁴

(中国科学院声学研究所语言声学与内容理解重点实验室北京100190)¹;
(中国科学院大学电子电气与通信工程学院北京100190)²;
(中国科学院新疆理化技术研究所新疆民族语音语言信息处理重点实验室乌鲁木齐 830011)³;
(贵州大学现代制造技术教育部重点实验室贵阳 550025)⁴

收稿日期:2019-05-31 出版日期:2020-03-15 发布日期:2020-03-30
通讯作者: 王冰(ariel.bingwang@outlook.com)
基金资助:
国家自然科学基金(11590770-4,11722437,61650202,U1536117,61671442,11674352,11504406,61601453);国家重点研发计划(2016YFB0801203,2016YFC0800503,2017YFB1002803);新疆维吾尔自治区重大科技专项(2016A03007-1);贵州省留学回国人员科技活动择优资助项目(2018.0002)

Session-based Recommendation Algorithm Based on Recurrent Temporal Convolutional Network

LI Tai-song^1,2,HE Ze-yu^1,2,WANG Bing^1,2,YAN Yong-hong^1,2,3,TANG Xiang-hong⁴

(Key Laboratory of Speech Acoustics and Content Understanding, Institute of Acoustics, Chinese Academy of Sciences, Beijing 100190, China)¹;
(School of Electronic, Electrical and Communication Engineering, University of Chinese Academy Sciences, Beijing 100190, China)²;
(Xinjiang Key Laboratory of Minority Speech and Language Information Processing, Xinjiang Technical Institute of Physics and Chemistry, Chinese Academy of Sciences, Urumchi 830011, China)³;
(Key Laboratory of Advanced Manufacturing Technology of the Ministry of Education, Guizhou University, Guiyang 550025, China)⁴

Received:2019-05-31 Online:2020-03-15 Published:2020-03-30
About author:LI Tai-song,born in 1990,doctorial student.His main research interests include recommendation system and data mining. WANG Bing,born in 1984,Ph.D,asso-ciate professor.Her main research interests include recommendation system and data mining.
Supported by:
This work was supported by the National Natural Science Foundation of China (11590770-4, 11722437, 61650202, U1536117, 61671442, 11674352, 11504406, 61601453), National Key Research and Development Program (2016YFB0801203, 2016YFC0800503, 2017YFB1002803), Key Science and Technology Project of the Xinjiang Uygur Autonomous Region (2016A03007-1) and Project of Guizhou High-level Study Abroad Talents Innovation and Entrepreneurship (2018.0002).

摘要/Abstract

摘要： 针对循环神经网络(Recurrent Neural Network,RNN)模型在序列流推荐中只能从宏观上捕捉序列的演变模式,忽略了物品(Item)间内部的微观联系,无法长程建模序列数据的变化规律的问题,提出了多维度序列建模算法循环时间卷积网络(Recurrent Temporal Convolutional Network,RTCN)。首先,将每个物品表示成定长向量,采用多层因果卷积和扩张卷积操作扩大感受野范围,建立序列元素间的长程依赖关系。利用残差连接网络提取不同层次的特征信息,解决反向传播中梯度衰减甚至消失的问题。综合设计时间卷积网络(Temporal Convolutional Network,TCN)提取序列流中前后物品间的局部特征,将物品信息映射到隐藏空间,得到细粒度的特征向量。为进一步建立元素间的宏观联系,将特征向量依次输入门限循环单元(Gated Recurrent Unit,GRU),迭代更新现有隐藏状态并预测下一时刻的输出。RTCN通过时间卷积网络,从输入序列流提取出长时间、多维度、细粒度的局部关联特征;经过门限循环网络,建模序列间的长距离依赖关系,捕捉序列元素的演变模式,并预测下一个出现的物品。利用网站、手机应用和音乐3个不同场景中的数据对模型进行了实验。实验结果显示,RTCN模型在召回率(Recall)和平均排序倒数(MRR)两个指标上比RNN模型高出6%~13%,比传统推荐算法高出9%~59%。通过对比不同的损失函数,模型在交叉熵损失函数下表现最优。此外,由于TCN中的卷积层具有多通道的结构,当数据维度丰富时,该模型对物品和用户的上下文信息具有很强的综合能力。

Abstract: Since the Recurrent Neural Network (RNN) generally models transition patterns,ignores the inner connection of items and can’t model the long-term evolving patterns of sequential data in session-based recommendations.A Recurrent Temporal Convolutional Network (RTCN) was proposed.Firstly,each item in the sequence is embedded as a vector,the multi-layer casual convolutions and dilated convolutions are applied so that the receptive field is improved and the long-term connections are established.A residual network is stacked to extract features from different layers.Therefore,the gradient vanishing or even disappearing in back propagation can be solved.With above operations,a well-designed Temporal Convolutional Network (TCN) is established.It extracts local features from sequence items,maps item information into latent space and generates fine-grained feature vectors as results.To further explore the connections between items in macroscopic way,the feature vectors are feed into Gated Recurrent Unit (GRU).After multiple iterations and updates to hidden states,the model can make a prediction of the next item.RTCN can extract long-time,multi-dimension,fine-grained local features from inputs by adapting temporal convolutional network.It also models the long-distance connections between items,captures the transition patterns and infers the next items by using GRU networks.The experimental results demonstrate that the RTCN model outperforms 6%～13% than RNN-based model and 9%～59% than other traditional recommendation methods under the metrics of Recall and Mean Reciprocal Rank (MRR).By comparing different definitions of loss,RTCN performs best under the cross entropy loss function.Meanwhile,due to the TCN multi-channel structure,the proposed model has a high potential capacity to embedding context features of items and users when the dataset information is rich.

Key words: Recommendation system, Deep learning, Session-based recommendation, Temporal convolutional network, Recurrent neural network

中图分类号:

TP183

李太松,贺泽宇,王冰,颜永红,唐向红. 基于循环时间卷积网络的序列流推荐算法[J]. 计算机科学, 2020, 47(3): 103-109.

LI Tai-song,HE Ze-yu,WANG Bing,YAN Yong-hong,TANG Xiang-hong. Session-based Recommendation Algorithm Based on Recurrent Temporal Convolutional Network[J]. Computer Science, 2020, 47(3): 103-109.

参考文献

[1]HIDASI B,KARATZOGLOU A,BALTRUNAS L,et al.Session-based recommendations with recurrent neural networks[J].arXiv:1511.06939,2015.
[2]RUMELHART D E,HINTON G E,WILLIAMS R J.Learning representations by back-propagating errors[J].Nature,1986,323(6088):399-421.
[3]KOREN Y,BELL R,VOLINSKY C.Matrix factorization tech- niques for recommender systems[J].Computer,2009,42(8):30-37.
[4]WEIMER M,KARATZOGLOU A.Cofi rank-maximum margin matrix factorization for collaborative ranking[C]∥Proceedings of the Twenty-First Annual Conference on Neural Information Processing Systems.2008:1593-1600.
[5]HIDASI B,TIKK D.Fast ALS-Based tensor factorization for context-aware recommendation from implicit feedback[C]∥Joint European Conference on Machine Learning and Knowledge Discovery in Databases.Berlin:Springer,2012:67-82.
[6]SARWAR B,KARYPIS G,KONSTAN J,et al.Item-based collaborative filtering recommendation algorithms[C]∥International Conference on World Wide Web.ACM,2001:285-295.
[7]KOREN Y.Factorization meets theneighborhood:a multifaceted collaborative filtering model[C]∥ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.ACM,2008:426-434.
[8]HIDASI B,QUADRANA M,TIKK D.Parallel recurrent neural network architectures for feature-rich session-based recommendations[C]∥ACM Conference on Recommender Systems.ACM,2016:241-248.
[9]BOGINA V,KUFLIK T.Incorporating dwell time in session- based recommendations with recurrent Neural networks [C]∥CEUR Workshop Proceedings.2017:57-59.
[10]QUADRANA M,KARATZOGLOU A,HIDASI B,et al.Personalizing session-based recommendations with hierarchical recurrent neural networks[C]∥Eleventh ACM Conference on Recommender Systems.ACM,2017:130-137.
[11]BAI S,KOLTER J Z,KOLTUN V.An empirical evaluation of generic convolutional and recurrent networks for sequence mo- deling [J].arXiv:1803.01271,2018.
[12]LIANG M,HU X.Recurrent convolutional neural network for object recognition[C]∥Computer Vision and Pattern Recognition.IEEE,2015:3367-3375.
[13]PINHEIRO P H O,COLLOBERT R.Recurrent convolutional neural networks for scene labeling[C]∥InternationalConfe-rence on International Conference on Machine Learning.2014:82-90.
[14]LÉCUN Y,BOTTOU L,BENGIO Y,et al.Gradient-based learning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[15]LONG J,SHELHAMER E,DARRELL T.Fully convolutional networks for semantic segmentation[C]∥IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer So-ciety,2015:3431-3440.
[16]HE K,ZHANG X,REN S,et al.Deep residual learning for ima- ge recognition[C]∥IEEE Conference on Computer Vision and Pattern Recognition.IEEE Computer Society,2016:770-778.
[17]SALIMANS T,KINGMA D P.Weight normalization:A simple reparameterization to accelerate training of deep neural networks[C]∥Advances in Neural Information Processing Systems.2016:901-909.
[18]CHUNG J,GULCEHRE C,CHO K H,et al.Empirical evaluation of gated recurrent neural networks on sequence modeling[J].arXiv:1412.3555,2014.
[19]BEN-SHIMON D,TSIKINOVSKY A,FRIEDMANN M,et al.Recsys challenge 2015 and the yoochoose dataset[C]∥RecSys’15:Proceedings of the 9th ACM Conference on Recommender Systems.New York:ACM,2015:357-358.
[20]CHO E,MYERS S A,LESKOVEC J.Friendship and mobility:user movement in locationbased social networks[C]∥Procee-dings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.ACM,2011:1082-1090.
[21]CELMA O.Music Recommendation and Discovery in the Long Tail[M].Springer,2010.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

基于循环时间卷积网络的序列流推荐算法

Session-based Recommendation Algorithm Based on Recurrent Temporal Convolutional Network

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

Metrics

本文评价

推荐阅读 10

[1]	李亚男, 胡宇佳, 甘伟, 朱敏. 基于深度学习的miRNA靶位点预测研究综述[J]. 计算机科学, 2021, 48(1): 209-216.
[2]	王瑞平, 贾真, 刘畅, 陈泽威, 李天瑞. 基于DeepFM的深度兴趣因子分解机网络[J]. 计算机科学, 2021, 48(1): 226-232.
[3]	于文家, 丁世飞. 基于自注意力机制的条件生成对抗网络[J]. 计算机科学, 2021, 48(1): 241-246.
[4]	仝鑫, 王斌君, 王润正, 潘孝勤. 面向自然语言处理的深度学习对抗样本综述[J]. 计算机科学, 2021, 48(1): 258-267.
[5]	丁钰, 魏浩, 潘志松, 刘鑫. 网络表示学习算法综述[J]. 计算机科学, 2020, 47(9): 52-59.
[6]	庄世杰, 於志勇, 郭文忠, 黄昉菀. 基于Zoneout的跨尺度循环神经网络及其在短期电力负荷预测中的应用[J]. 计算机科学, 2020, 47(9): 105-109.
[7]	何鑫, 许娟, 金莹莹. 行为关联网络:完整的变化行为建模[J]. 计算机科学, 2020, 47(9): 123-128.
[8]	叶亚男, 迟静, 于志平, 战玉丽, 张彩明. 基于改进CycleGan模型和区域分割的表情动画合成[J]. 计算机科学, 2020, 47(9): 142-149.
[9]	邓良, 许庚林, 李梦杰, 陈章进. 基于深度学习与多哈希相似度加权实现快速人脸识别[J]. 计算机科学, 2020, 47(9): 163-168.
[10]	游兰, 韩雪薇, 何正伟, 肖丝雨, 何渡, 潘筱萌. 基于改进Seq2Seq的短时AIS轨迹序列预测模型[J]. 计算机科学, 2020, 47(9): 169-174.
[11]	暴雨轩, 芦天亮, 杜彦辉. 深度伪造视频检测技术综述[J]. 计算机科学, 2020, 47(9): 283-292.
[12]	袁野, 和晓歌, 朱定坤, 王富利, 谢浩然, 汪俊, 魏明强, 郭延文. 视觉图像显著性检测综述[J]. 计算机科学, 2020, 47(7): 84-91.
[13]	王文刀, 王润泽, 魏鑫磊, 漆云亮, 马义德. 基于堆叠式双向LSTM的心电图自动识别算法[J]. 计算机科学, 2020, 47(7): 118-124.
[14]	刘燕, 温静. 基于注意力机制的复杂场景文本检测[J]. 计算机科学, 2020, 47(7): 135-140.
[15]	张志扬, 张凤荔, 谭琪, 王瑞锦. 基于深度学习的信息级联预测方法综述[J]. 计算机科学, 2020, 47(7): 141-153.