计算机科学 ›› 2022, Vol. 49 ›› Issue (1): 101-107.doi: 10.11896/jsjkx.201200007

所属专题: 大数据&数据科学 虚拟专题

• 数据库&大数据&数据科学 • 上一篇    下一篇

基于DeepFM和卷积神经网络的集成式多模态谣言检测方法

陈志毅, 隋杰   

  1. 中国科学院大学工程科学学院 北京100049
  • 收稿日期:2020-12-01 修回日期:2021-04-16 出版日期:2022-01-15 发布日期:2022-01-18
  • 通讯作者: 隋杰(suijie@ucas.ac.cn)
  • 作者简介:18811722686@163.com
  • 基金资助:
    国家重点研发计划(2017YFB0803001);国家自然科学基金(61572459)

DeepFM and Convolutional Neural Networks Ensembles for Multimodal Rumor Detection

CHEN Zhi-yi, SUI Jie   

  1. School of Engineering Science,University of Chinese Academy of Sciences,Beijing 100049,China
  • Received:2020-12-01 Revised:2021-04-16 Online:2022-01-15 Published:2022-01-18
  • About author:CHEN Zhi-yi,born in 1996,postgra-duate.His main research interests include natural language processing and data mining.
    SUI Jie,born in 1976,associate professor.Her main research interests include data mining and social network analysis.
  • Supported by:
    National Key R & D Program of China(2017YFB0803001) and National Natural Science Foundation of China(61572459).

摘要: 随着以微博为代表的社交媒体越来越流行,谣言信息借助社交媒体迅速传播,容易造成严重的后果,因此自动谣言检测问题受到了国内外学术界、产业界的广泛关注。目前,越来越多的用户使用图片来发布微博,而不仅仅是文本,微博通常由文本、图像和社会语境组成。因此,文中提出了一种基于深度神经网络,针对配文文本内容、图像以及用户属性信息的多模态网络谣言检测方法DCNN。该方法由多模态特征提取器和谣言检测器组成,多模态特征提取器分为3部分,即基于TextCNN的文本特征提取器、基于VGG-19的图片特征提取器和基于DeepFM算法的用户社会特征提取器,分别用于学习微博不同模态上的特征表示,以形成重新参数化的多模态特征,特征融合后将该融合后的多模态特征作为谣言检测器的输入进行分类检测。在微博数据集上对该算法进行了大量实验,实验结果表明DCNN算法将识别准确率从78.1%提高到了80.3%,验证了DCNN算法和其中对社会特征建立特征交互方法的可行性与有效性。

关键词: DeepFM, 多模态, 卷积神经网络, 社会特征, 谣言检测, 自然语言处理

Abstract: With the increasing popularity of social media represented by Weibo,rumors spread rapidly through social media,which is more likely to cause serious consequences.The problem of automatic rumor detection has attracted widespread attention from academic and industrial circles at home and abroad.We have noticed that more and more users use pictures to post Weibo,not just text.Weibo usually consists of text,images and social context.Therefore,a multi-modal network rumor detection method DCNN based on deep neural network for the text content,image and user attribute information of the accompanying text is proposed.This method consists of a multi-modal feature extractor and a rumor detector.The multi-modal feature extractor is divided into three parts:a text feature extractor based on TextCNN,a picture feature extractor based on VGG-19,and a user social feature extractor based on DeepFM algorithm.These three parts learn feature representations on different modalities of Weibo to form re-parameterized multi-modal features,which are fused as input to the rumor detector classification detection.This algorithm has carried out a large number of experiments on the Weibo data set,and the experimental results show that the recognition accuracy of DCNN algorithm is improved from 78.1% to 80.3%,which verifies the feasibility and effectiveness of DCNN algorithm and feature interaction method for social characteristics.

Key words: Convolutional neural networks, DeepFM, Multimodal, Natural language processing, Rumor detection, Social feature

中图分类号: 

  • TP391
[1]JIN Z W,CAO J,ZHANG Y D.Novel Visual and StatisticalImage Features for Microblogs News Verification[J].IEEE Transactions on Multimedia,2016,19(3):598-608.
[2]CARLOS C,MARCELO M,BARBARA P.Information credibi-lity on twitter[C]//20th International Conference on World Wide Web (WWW).ACM,2011:675-684.
[3]SALLOUM R,REN Y,KUO C C J.Image splicing localization using a multi-task fully convolutional network (MFCN)[J].Journal of Visual Communication and Image Representation,2018,51:201-208.
[4]WU K,YANG S,ZHU K Q.False rumors detection on sinaweibo by propagation structures [C]//IEEE International Confe-rence on Data Engineering.ICDE,2015:651-662.
[5]MANISH G,ZHAO P,HAN J W.Evaluating Event Credibility on Twitter[C]//SIAM International Conference on Data Mi-ning.Society for Industrial and Applied Mathematics,2012:153-164.
[6]JIN Z W,CAO J,JIANG Y G,et al.News credibility evaluationon microblog with a hierarchical propagation model [C]//IEEE International Conference on Data Mining (ICDM).IEEE,2014:230-239.
[7]JIN Z W,CAO J,ZHANG Y D,et al.News Verification by Exploiting Conflicting Social Viewpoints in Microblogs[C]//Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence.2016:2-17.
[8]ADITI G,HEMANK L,PONNURANGAM K,et al.FakingSandy:characterizing and identifying fake images on Twitter during Hurricane Sandy[C]//Proceedings of the 22nd International Conference on World Wide Web Companion.2013:729-736.
[9]JIN Z W,CAO J,ZHANG Y Z,et al.Verifying Multimedia Use with a Two-Level Classification Model[C]//MediaEval Multimedia Benchmark Workshop.2015.
[10]YOON K.Convolutional Neural Networks for Sentence Classification[C]//2014 Conference on Empirical Methods in Natural Language Processing.2014.
[11]KAREN S,ANDREW Z.Very Deep ConvNets for Large-Scale Image Recognition[C]//3rd International Conference on Lear-ning Representations.ACM,2015:358-406.
[12]SUN S,LIU H,HE J,et al. Detecting event rumors on sinaweibo automatically[C]//Web Technologies and Applications.Springer,2013:120-131.
[13]YANG F,LIU Y,YU X H,et al.Automatic detection of rumor on sina weibo[C]//Proceedings of the ACM SIGKDD Workshop on Mining Data Semantics.2012:1-7.
[14]KWON S,CHA M,JUNG K,et al. Prominent features of rumor propagation in online social media[C]//IEEE 13th International Conference on Data Mining (ICDM).IEEE,2013:1103-1108.
[15]MA J,GAO W,MITRA P,et al.Detecting rumors from micro-blogs with recurrent neural networks [C]//International Joint Conference on Artificial Intelligence.2016:3818-3824.
[16]CHEN T,LI X,ZHANG J,et al.Call attention to rumors:Deep attention based recurrent neural networks for early rumor detection[C]//Trends and Applications in Knowledge Discovery and Data.2017:40-52.
[17]RUCHANSKY N,SEO S,LIU Y.CSI:A hybrid deep model for fake news detection[C]//Conference on Information and Knowledge Management.2017:797-806.
[18]NGUYENT N,LI C,NIEDERÈE C.On early-stage debunking rumors on twitter:Leveraging the wisdom of weak learners [C]//International Conference on Social Informatics.Springer,2017:141-158.
[19]JIN Z W,CAO J,GUO H,et al.Multimodal fusion with recurrent neural networks for rumor detection on microblogs[C]//ACM on Multimedia Conference.2017:795-816.
[20]WANG Y Q,MA F L,JIN Z W,et al.Eann:Event adversarial neural networks for multi-modal fake news detection[C]//24th ACM SIGKDD International Conference on Knowledge Disco-very & Data Mining.2018:849-857.
[21]KHATTAR D,GOUD J S,GUPTA M,et al.Mvae:Multimodal variational autoencoder for fake news detection [C]//World Wide Web Conference.ACM,2019:2915-2921.
[22]WANG R P,JIA Z,LIU C,et al.Deep Interest FactorizationMachine Network Based on DeepFM[J].Computer Science,2021,48(1):226-232.
[23]STANISLAW A,AISHWARYA A,JIASEN L,et al.Vqa:Vi-sual question answering [C]//IEEE International Conference on Computer Vision.2015:2425-2433.
[24]LIU J S,FENG K,JEFF Z,et al.MSRD:Multi-Modal Web Rumor Detection Method[J].Journal of Computer Research and Development,2020,57(11):2328-2336.
[1] 聂秀山, 潘嘉男, 谭智方, 刘新放, 郭杰, 尹义龙.
基于自然语言的视频片段定位综述
Overview of Natural Language Video Localization
计算机科学, 2022, 49(9): 111-122. https://doi.org/10.11896/jsjkx.220500130
[2] 周旭, 钱胜胜, 李章明, 方全, 徐常胜.
基于对偶变分多模态注意力网络的不完备社会事件分类方法
Dual Variational Multi-modal Attention Network for Incomplete Social Event Classification
计算机科学, 2022, 49(9): 132-138. https://doi.org/10.11896/jsjkx.220600022
[3] 周乐员, 张剑华, 袁甜甜, 陈胜勇.
多层注意力机制融合的序列到序列中国连续手语识别和翻译
Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion
计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026
[4] 李宗民, 张玉鹏, 刘玉杰, 李华.
基于可变形图卷积的点云表征学习
Deformable Graph Convolutional Networks Based Point Cloud Representation Learning
计算机科学, 2022, 49(8): 273-278. https://doi.org/10.11896/jsjkx.210900023
[5] 姜梦函, 李邵梅, 郑洪浩, 张建朋.
基于改进位置编码的谣言检测模型
Rumor Detection Model Based on Improved Position Embedding
计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[6] 陈泳全, 姜瑛.
基于卷积神经网络的APP用户行为分析方法
Analysis Method of APP User Behavior Based on Convolutional Neural Network
计算机科学, 2022, 49(8): 78-85. https://doi.org/10.11896/jsjkx.210700121
[7] 朱承璋, 黄嘉儿, 肖亚龙, 王晗, 邹北骥.
基于注意力机制的医学影像深度哈希检索算法
Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism
计算机科学, 2022, 49(8): 113-119. https://doi.org/10.11896/jsjkx.210700153
[8] 檀莹莹, 王俊丽, 张超波.
基于图卷积神经网络的文本分类方法研究综述
Review of Text Classification Methods Based on Graph Convolutional Network
计算机科学, 2022, 49(8): 205-216. https://doi.org/10.11896/jsjkx.210800064
[9] 闫佳丹, 贾彩燕.
基于双图神经网络信息融合的文本分类方法
Text Classification Method Based on Information Fusion of Dual-graph Neural Network
计算机科学, 2022, 49(8): 230-236. https://doi.org/10.11896/jsjkx.210600042
[10] 张颖涛, 张杰, 张睿, 张文强.
全局信息引导的真实图像风格迁移
Photorealistic Style Transfer Guided by Global Information
计算机科学, 2022, 49(7): 100-105. https://doi.org/10.11896/jsjkx.210600036
[11] 戴朝霞, 李锦欣, 张向东, 徐旭, 梅林, 张亮.
基于DNGAN的磁共振图像超分辨率重建算法
Super-resolution Reconstruction of MRI Based on DNGAN
计算机科学, 2022, 49(7): 113-119. https://doi.org/10.11896/jsjkx.210600105
[12] 刘月红, 牛少华, 神显豪.
基于卷积神经网络的虚拟现实视频帧内预测编码
Virtual Reality Video Intraframe Prediction Coding Based on Convolutional Neural Network
计算机科学, 2022, 49(7): 127-131. https://doi.org/10.11896/jsjkx.211100179
[13] 徐鸣珂, 张帆.
Head Fusion:一种提高语音情绪识别的准确性和鲁棒性的方法
Head Fusion:A Method to Improve Accuracy and Robustness of Speech Emotion Recognition
计算机科学, 2022, 49(7): 132-141. https://doi.org/10.11896/jsjkx.210100085
[14] 侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木.
中文预训练模型研究进展
Advances in Chinese Pre-training Models
计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[15] 金方焱, 王秀利.
融合RACNN和BiLSTM的金融领域事件隐式因果关系抽取
Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM
计算机科学, 2022, 49(7): 179-186. https://doi.org/10.11896/jsjkx.210500190
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!