计算机科学 ›› 2021, Vol. 48 ›› Issue (5): 117-123.doi: 10.11896/jsjkx.200400057
所属专题: 大数据&数据科学 虚拟专题
张少钦1, 杜圣东1,2,3, 张晓博1,2,3, 李天瑞1,2,3
ZHANG Shao-qin1, DU Sheng-dong1,2,3, ZHANG Xiao-bo1,2,3, LI Tian-rui1,2,3
摘要: 随着社交网络平台的发展,社交网络已经成为人们获取信息的重要来源。然而社交网络的便利性也导致了虚假谣言的快速传播。与纯文本的谣言相比,带有多媒体信息的网络谣言更容易误导用户以及被传播,因此对多模态的网络谣言检测在现实生活中有着重要意义。研究者们已提出若干多模态的网络谣言检测方法,但这些方法都没有充分挖掘出视觉特征和融合文本与视觉的联合表征特征。为弥补这些不足,提出了一个基于深度学习的端到端的多模态融合网络。该网络首先抽取出图片中各个兴趣区域的视觉特征,然后使用多头注意力机制将文本和视觉特征进行更新与融合,最后将这些特征进行基于注意力机制的拼接以用于社交网络多模态谣言检测。在推特和微博公开数据集上进行对比实验,结果表明,所提方法在推特数据集上F1值有13.4%的提升,在微博数据集上F1值有1.6%的提升。
中图分类号:
[1]CHEN Y F,LI Z Y,LIANG X,et al.Review on Rumor Detection of Online Social Networks[J].Chinese Journal of Compu-ters,2018,41(7):1648-1677. [2]KARPATHY A,LI F F.Deep Visual-Semantic Alignments for Generating Image Descriptions[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(4):664-676. [3]ANDERSON P,HE X,BUEHLER C,et al.Bottom-Up andTop-Down Attention for Image Captioning and Visual Question Answering[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.ACM,2018:6077-6086. [4]GAO P,JIANG Z,YOU H,et al.Dynamic fusion with intra-and inter-modality attention flow for visual question answering[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.ACM,2019:6632-6641. [5]BALTRUSAITIS T,AHUJA C,MORENCY L P.Multimodal Machine Learning:A Survey and Taxonomy[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2019,41(2):423-443. [6]CASTILLO C,MENDOZA M,POBLETE B.Predicting infor-mation credibility in time-sensitive social media[J].Internet Research,2013,23(5):560-588. [7]ZHAO Z,RESNICK P,MEI Q.Enquiring minds:Early detection of rumors in social media from enquiry posts[C]//Procee-dings of the 24th International Conference on World Wide Web.2015:1395-1405. [8]TSCHIATSCHEK S,SINGLA A,GOMEZ R M,et al.FakeNews Detection in Social Networks via Crowd Signals[C]//Companion of The Web Conference 2018 on The Web Confe-rence 2018.2018:517-524. [9]GUPTA A,LAMBA H,KUMARAGURU P,et al.Fakingsandy:characterizing and identifying fake images on twitter during hurricane sandy[C]//Proceedings of the 22nd International Conference on World Wide Web.ACM,2013:729-736. [10]JIN Z,CAO J,ZHANG Y,et al.Novel visual and statistical ima-ge features for microblogs news verification[J].IEEE Transactions on Multimedia,2017,19(3):598-608. [11]LIU Z,WEI Z H,ZHANG R X.Rumor detection based on con-volutional neural network[J].Journal of Computer Applications,2017(11):21-24,68. [12]LIAO X W,HUANG Z,YANG D D,et al.Rumor detection in social media based on a hierarchical attention network[J].Scientia Sinica Informationis,2018,48(11):1558-1574. [13]MA J,GAO W,WONG K.Rumor detection on twitter withtree-structured recursive neural networks[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.2018:1980-1989. [14]RUCHANSKY N,SEO S,LIU Y.CSI:A hybrid deep model for fake news detection[C]//Proceedings of the 2017 ACM on Conference on Information and Knowledge Management.ACM,2017:797-806. [15]HOCHREITER S,SCHMIDHUBER J.Long Short-Term Me-mory[J].Neural Computation,1997,9(8):1735-1780. [16]ZHANG J,DONG B,YU P S.Deep Diffusive Neural Network based Fake News Detection from Heterogeneous Social Networks[C]//Proceedings of IEEE International Conference on Big Data.2019:1259-1266. [17]SHU K,WANG S,LIU H.Beyond news contents:The role of social context for fake news detection[C]//Proceedings of the 12th ACM International Conference on Web Search and Data Mining.2019:312-320. [18]LIU Y,WU Y.Early detection of fake news on social media through propagation path classification with recurrent andcon-volutional networks[C]//Proceedings of Thirty-Second AAAI Conference on Artificial Intelligence.2018:354-361. [19]MIKOLOV T,KARAFIAT M,BURGET L,et al.Recurrentneural network based language model[C]//Proceedings of 11th Annual Conference of the International Speech Communication Association.2010:1045-1048. [20]YANG Y,ZHENG L,ZHANG J,et al.TI-CNN:Convolutional Neural Networks for Fake News Detection[J].arXiv:1806.00749,2018. [21]JIN Z,CAO J,HAN G,et al.Multimodal Fusion with Recurrent Neural Networks for Rumor Detection on Microblogs[C]//Proceedings of the 2017 ACM on Multimedia Conference.ACM,2017:795-816. [22]WANG Y,MA F,JIN Z,et al.EANN:Event Adversarial Neural Networks for Multi-Modal Fake News Detection[C]//Procee-dings of The 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.ACM,2018:849-857. [23]SCHUSTER M,PALIWAL K.Bidirectional recurrent neuralnetworks[J].IEEE Transactions on Signal Processing,1997,45(11):2673-2681. [24]KIM Y.Convolutional Neural Networks for Sentence Classification[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing(EMNLP).ACM,2014:1746-1751. [25]GIRSHICK R,DONAHUE J,DARRELL T,et al.Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of the IEEE Computer Society Confe-rence on Computer Vision and Pattern Recognition.ACM 2014:580-587. [26]REN S,HE K,GIRSHICK R,et al.Faster R-CNN:Towards Real-Time Object Detection with Region Proposal Networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017(6):1137-1149. [27]VASWANI A,SHAZEER N,PARMAR N,et al.Attention IsAll You Need[J].Advances in Neural Information Processing Systems,2017(12):5999-6009. [28]GU Y,YANG K,FU S,et al.Hybrid Attention based Multimodal Network for Spoken Language Classification[C]//Procee-dings of the 27th International Conference on Computational Linguistics.ACM,2018:2379-2390. [29]BOIDIDOU C,PAPADOPOULOS S,ZAMPOGLOU M,et al.Detection and visualization of misleading content on Twitter[J].International Journal of Multimedia Information Retrieval,2018,7(1):71-86. [30]SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-Scale Image Recognition[C]//Proceedings of International Conference on Learning Representations.ICLR,2015:1-14. |
[1] | 饶志双, 贾真, 张凡, 李天瑞. 基于Key-Value关联记忆网络的知识图谱问答方法 Key-Value Relational Memory Networks for Question Answering over Knowledge Graph 计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277 |
[2] | 汤凌韬, 王迪, 张鲁飞, 刘盛云. 基于安全多方计算和差分隐私的联邦学习方案 Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy 计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108 |
[3] | 聂秀山, 潘嘉男, 谭智方, 刘新放, 郭杰, 尹义龙. 基于自然语言的视频片段定位综述 Overview of Natural Language Video Localization 计算机科学, 2022, 49(9): 111-122. https://doi.org/10.11896/jsjkx.220500130 |
[4] | 周旭, 钱胜胜, 李章明, 方全, 徐常胜. 基于对偶变分多模态注意力网络的不完备社会事件分类方法 Dual Variational Multi-modal Attention Network for Incomplete Social Event Classification 计算机科学, 2022, 49(9): 132-138. https://doi.org/10.11896/jsjkx.220600022 |
[5] | 徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺. 时序知识图谱表示学习 Temporal Knowledge Graph Representation Learning 计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204 |
[6] | 王剑, 彭雨琦, 赵宇斐, 杨健. 基于深度学习的社交网络舆情信息抽取方法综述 Survey of Social Network Public Opinion Information Extraction Based on Deep Learning 计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099 |
[7] | 郝志荣, 陈龙, 黄嘉成. 面向文本分类的类别区分式通用对抗攻击方法 Class Discriminative Universal Adversarial Attack for Text Classification 计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077 |
[8] | 姜梦函, 李邵梅, 郑洪浩, 张建朋. 基于改进位置编码的谣言检测模型 Rumor Detection Model Based on Improved Position Embedding 计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046 |
[9] | 刘冬梅, 徐洋, 吴泽彬, 刘倩, 宋斌, 韦志辉. 基于边框距离度量的增量目标检测方法 Incremental Object Detection Method Based on Border Distance Measurement 计算机科学, 2022, 49(8): 136-142. https://doi.org/10.11896/jsjkx.220100132 |
[10] | 王灿, 刘永坚, 解庆, 马艳春. 基于软标签和样本权重优化的Anchor Free目标检测算法 Anchor Free Object Detection Algorithm Based on Soft Label and Sample Weight Optimization 计算机科学, 2022, 49(8): 157-164. https://doi.org/10.11896/jsjkx.210600240 |
[11] | 孙奇, 吉根林, 张杰. 基于非局部注意力生成对抗网络的视频异常事件检测方法 Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection 计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061 |
[12] | 胡艳羽, 赵龙, 董祥军. 一种用于癌症分类的两阶段深度特征选择提取算法 Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification 计算机科学, 2022, 49(7): 73-78. https://doi.org/10.11896/jsjkx.210500092 |
[13] | 程成, 降爱莲. 基于多路径特征提取的实时语义分割方法 Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction 计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157 |
[14] | 侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木. 中文预训练模型研究进展 Advances in Chinese Pre-training Models 计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018 |
[15] | 周慧, 施皓晨, 屠要峰, 黄圣君. 基于主动采样的深度鲁棒神经网络学习 Robust Deep Neural Network Learning Based on Active Sampling 计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044 |
|