计算机科学 ›› 2020, Vol. 47 ›› Issue (6): 201-209.doi: 10.11896/jsjkx.200200117

• 人工智能 • 上一篇    下一篇

基于分层注意力的信息级联预测模型

张志扬, 张凤荔, 陈学勤, 王瑞锦   

  1. 电子科技大学信息与软件工程学院 成都610054
  • 收稿日期:2020-02-26 出版日期:2020-06-15 发布日期:2020-06-10
  • 通讯作者: 张凤荔(fzhang@uestc.edu.cn)
  • 作者简介:13980044734@163.com
  • 基金资助:
    国家自然科学基金(61802033,61472064,61602096);四川省科技计划(2018GZ0087,2019YJ0543);博士后基金项目(2018M643453);广东省国家重点实验室项目(2017B030314131);网络与数据安全四川省重点实验室开放课题(NDSMS201606)

Information Cascade Prediction Model Based on Hierarchical Attention

ZHANG Zhi-yang, ZHANG Feng-li, CHEN Xue-qin, WANG Rui-jin   

  1. School of Information and Software Engineering,University of Electronic Science and Technology of China,Chengdu610054,China
  • Received:2020-02-26 Online:2020-06-15 Published:2020-06-10
  • About author:ZHANG Zhi-yang,born in 1997,postgraduate,is a member of China Computer Federation.His main research interests include machine learning,data mining and cascade prediction.
    ZHANG Feng-li,born in 1963,Ph.D,professor,Ph.D supervisor,is a member of China Computer Federation.Her main research interests include network security and network engineering,cloud computing and big data and machine learning.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China(61802033,61472064,61602096),Sichuan Science and Technology Program (2018GZ0087,2019YJ0543),Chinese Postdoctoral Science Foundation(2018M643453),Guangdong Provincial Key Laboratory Project(2017B030314131) and Network and Data Security Key Laboratory of Sichuan Province Open Issue(NDSMS201606).

摘要: 信息级联预测(Information Cascade Prediction)是社交网络分析领域的一个研究热点,其通过信息级联的扩散序列与拓扑图来学习在线社交媒体中信息的传播模式。当前的信息级联预测模型大多以循环神经网络为基础,仅考虑信息级联的时序结构信息或者序列内部的空间结构信息,无法学习序列之间的拓扑关系。而现有的级联图结构学习方法无法为节点的邻居分配不同的权重,导致节点之间的关联性学习较差。针对上述问题,文中提出了基于节点表示的信息级联采样方法,将信息级联建模为节点表示而非序列表示。随后提出一种基于分层注意力的信息级联预测(Information Cascade Prediction with Hierarchical Attention,ICPHA)模型。该模型首先通过结合了自注意力机制的循环神经网络来学习节点序列的时序结构信息;然后通过多头注意力机制学习节点表示之间的空间结构信息;最后通过分层的注意力网络对信息级联的结构信息进行联合建模。所提模型在Twitter,Memes,Digg这3种数据集上达到了领先的预测效果,并且具有良好的泛化能力。

关键词: 多头注意力机制, 深度学习, 图表示学习, 信息级联预测, 循环神经网络, 在线社交媒体

Abstract: Information cascade prediction is a research hotspot in the field of social network analysis.It learns the propagation mode of information in online social media through the diffusion sequence and topology map of the information cascade.Most current models for solving this task are based on recurrent neural networks and only consider information cascading time series structure information or spatial structure information inside sequences,and cannot learn topological relationships between sequences.And the existing cascade graph structure learning methods cannot assign different weights to the neighbors of the nodes,resulting in poor association learning between the nodes.In response to the above problems,this paper proposes an information cascade sampling method based on node representation,which models the information cascade as a node representation rather than a sequence representation.This paper also proposes an information cascade prediction model based on hierarchical attention network (ICPHA),which learns the time series structure information of the node sequence through a recurrent neural network layer with self-attention mechanism,and learns the spatial structure information between node representations through a multi-head attention mechanism.By this way,ICPHA jointly models the structural information of the information cascade through a hierarchical attention network.ICPHA has achieved leading prediction results on Twitter,Memes,and Digg,and has good generalization ability.

Key words: Deep learning, Graph representation learning, Information cascade prediction, Multi-head attention mechanism, Online social media, Recurrent neural network

中图分类号: 

  • TP183
[1]ZHU X,JIA Y,NIE Y P,et al.Event Propagation Analysis on Microblog[J].Journal of Computer Research and Development,2015,52(2):437-444.
[2]CHENG J,ADAMIC L,DOW P A,et al.Can cascades be predicted?[C]//Proceedings of the 23rd International Conference on World Wide Web.ACM,2014:925-936.
[3]JIANG Y,COUNTS S.Predicting the speed,scale,and range of information diffusion in twitter[C]//Fourth International AAAI Conference on Weblogs and Social Media.2010.
[4]GOLUB B,JACKSON M O.Using selection bias to explain the observed structure of internet diffusions[J].Proceedings of the National Academy of Sciences,2010,107(24):10833-10836.
[5]LESKOVEC J.The Dynamics of Viral Marketing[J].Acm Transactions on the Web,2005,1(1):228-237.
[6]DOW A P,ADAMIC L A,FRIGGERI A.The Anatomy of Large Facebook Cascades[C]//ICWSM.2013.
[7]KUMAR R,MAHDIAN M,MCGLOHON M.Dynamics of conversations[C]//Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.ACM,2010:553-562.
[8]CHENG L,MA J Q,GUO X X,et al.Deepcas:An end-to-end predictor of information cascades[C]//Proceedings of the 26th international conference on World Wide Web.International World Wide Web Conferences Steering Committee,2017:577-586.
[9]WANG X S,MA S Z.Method of Weibo User Influence Calculation Integrating Users’ Own Factors and Interaction Behavior[J].Computer Science,2020,47(1):96-101.
[10]BENGIO Y,DUCHARME R,VINCENT P,et al.A neural probabilistic language model[J].Journal of Machine Learning Research,2003,3(Feb):1137-1155.
[11]KRIZHEVSKY A,SUTSKEVER I,HINTON G E.Imagenet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems.2012:1097-1105.
[12]QIU J Z,TANG J,MA H,et al.Deepinf:Social influence prediction with deep learning[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.ACM,2018:2110-2119.
[13]VELIKOVI P,CUCURULL G,CASANOVA A,et al.Graph attention networks[J].arXiv:1710.10903,2017.
[14]GUO R C,SHAKARIAN P.A comparison of methods for cascade prediction[C]//Proceedings of the 2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining.IEEE,2016:591-598.
[15]BAO P,SHEN H W,JIN X L,et al.Modeling and predicting popularity dynamics of microblogs using self-excited hawkes processes[C]//Proceedings of the 24th International Conference on World Wide Web.ACM,2015:9-10.
[16]KEMPE D,KLEINBERG J,TARDOS É.Maximizing the spread of influence through a social network[C]//Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.ACM,2003:137-146.
[17]CAO Q,SHEN H W,CEN K T,et al.Deephawkes:Bridging the gap between prediction and understanding of information cascades[C]//Proceedings of the 2017 ACM on Conference on Information and Knowledge Management.ACM,2017:1149-1158.
[18]CHEN X Q,ZHOU F,ZHANG K P,et al.Information Diffusion Prediction via Recurrent Cascades Convolution[C]//2019 IEEE 35th International Conference on Data Engineering (ICDE).IEEE,2019:770-781.
[19]KIPF T N,WELLING M.Semi-supervised classification withgraph convolutional networks[J].arXiv:1609.02907,2016.
[20]TONG H H,FALOUTSOS C,PAN J Y.Fast random walk with restart and its applications[C]//Sixth International Conference on Data Mining (ICDM’06).IEEE,2006:613-622.
[21]MNIH V,HEESS N,GRAVES A.Recurrent models of visual attention[C]//Advances in Neural Information Processing Systems.2014:2204-2212.
[22]WANG Z T,CHEN C Y,LI W J.A Sequential Neural Information Diffusion Model with Structure Attention[C]//Proceedings of the 27th ACM International Conference on Information and Knowledge Management.ACM,2018:1795-1798.
[23]ISLAM M R,MUTHIAH S,ADHIKARI B,et al.DeepDiffuse:Predicting the ‘Who’ and ‘When’ in Cascades[C]//2018 IEEE International Conference on Data Mining (ICDM).IEEE,2018:1055-1060.
[24]GAO S,MA J,CHEN Z M.Modeling and predicting retweeting dynamics on microblogging platforms[C]//Proceedings of the Eighth ACM International Conference on Web Search and Data Mining.ACM,2015:107-116.
[25]LESKOVEC J,BACKSTROM L,KLEINBERG J.Meme-tracking and the dynamics of the news cycle[C]//Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.ACM,2009:497-506.
[26]TAD H,KRISTINA L.Social Dynamics of Digg[C]//Proceedings of the Fourth International Conference on Weblogs and Social Media(ICWSM 2010).Washington,DC,USA,2010:23-26.
[27]WANG J,ZHENG V,LIU Z M,et al.Topological recurrentneural network for diffusion prediction[C]//2017 IEEE International Conference on Data Mining (ICDM).IEEE,2017:475-484.
[1] 宋杰, 梁美玉, 薛哲, 杜军平, 寇菲菲.
基于无监督集群级的科技论文异质图节点表示学习方法
Scientific Paper Heterogeneous Graph Node Representation Learning Method Based onUnsupervised Clustering Level
计算机科学, 2022, 49(9): 64-69. https://doi.org/10.11896/jsjkx.220500196
[2] 徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺.
时序知识图谱表示学习
Temporal Knowledge Graph Representation Learning
计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[3] 饶志双, 贾真, 张凡, 李天瑞.
基于Key-Value关联记忆网络的知识图谱问答方法
Key-Value Relational Memory Networks for Question Answering over Knowledge Graph
计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[4] 汤凌韬, 王迪, 张鲁飞, 刘盛云.
基于安全多方计算和差分隐私的联邦学习方案
Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy
计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[5] 王剑, 彭雨琦, 赵宇斐, 杨健.
基于深度学习的社交网络舆情信息抽取方法综述
Survey of Social Network Public Opinion Information Extraction Based on Deep Learning
计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[6] 郝志荣, 陈龙, 黄嘉成.
面向文本分类的类别区分式通用对抗攻击方法
Class Discriminative Universal Adversarial Attack for Text Classification
计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[7] 姜梦函, 李邵梅, 郑洪浩, 张建朋.
基于改进位置编码的谣言检测模型
Rumor Detection Model Based on Improved Position Embedding
计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[8] 孙奇, 吉根林, 张杰.
基于非局部注意力生成对抗网络的视频异常事件检测方法
Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection
计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[9] 胡艳羽, 赵龙, 董祥军.
一种用于癌症分类的两阶段深度特征选择提取算法
Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification
计算机科学, 2022, 49(7): 73-78. https://doi.org/10.11896/jsjkx.210500092
[10] 程成, 降爱莲.
基于多路径特征提取的实时语义分割方法
Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction
计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[11] 侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木.
中文预训练模型研究进展
Advances in Chinese Pre-training Models
计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[12] 周慧, 施皓晨, 屠要峰, 黄圣君.
基于主动采样的深度鲁棒神经网络学习
Robust Deep Neural Network Learning Based on Active Sampling
计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
[13] 苏丹宁, 曹桂涛, 王燕楠, 王宏, 任赫.
小样本雷达辐射源识别的深度学习方法综述
Survey of Deep Learning for Radar Emitter Identification Based on Small Sample
计算机科学, 2022, 49(7): 226-235. https://doi.org/10.11896/jsjkx.210600138
[14] 彭双, 伍江江, 陈浩, 杜春, 李军.
基于注意力神经网络的对地观测卫星星上自主任务规划方法
Satellite Onboard Observation Task Planning Based on Attention Neural Network
计算机科学, 2022, 49(7): 242-247. https://doi.org/10.11896/jsjkx.210500093
[15] 祝文韬, 兰先超, 罗唤霖, 岳彬, 汪洋.
改进Faster R-CNN的光学遥感飞机目标检测
Remote Sensing Aircraft Target Detection Based on Improved Faster R-CNN
计算机科学, 2022, 49(6A): 378-383. https://doi.org/10.11896/jsjkx.210300121
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!