Computer Science ›› 2024, Vol. 51 ›› Issue (4): 307-313.doi: 10.11896/jsjkx.230900087

• Artificial Intelligence • Previous Articles     Next Articles

Study on Improved Fake Information Detection Method Based on Cross-modal CorrelationAmbiguity Learning

DUAN Yuxiao, HU Yanli, GUO Hao, TAN Zhen, XIAO Weidong   

  1. Science and Technology on Information Systems Engineering Laboratory,National University of Defense Technology,Changsha 410073,China
  • Received:2023-09-15 Revised:2023-11-06 Online:2024-04-15 Published:2024-04-10
  • Supported by:
    National Natural Science Foundation of China(62272469,72301284),National Key R & D Program of China(2022YFB3102600) and Science and Technology Innovation Program of Hunan Province(2023RC1007).

Abstract: In recent years,with the rapid development of the Internet and multimedia technology,it is more convenient for people to obtain information,but the spread of fake information on the Internet is also increasingly serious,and the negative impact is constantly expanding.In order to enhance the credibility and deception,fake information presents a multi-modal development trend,which makes the detection work face greater challenges.The existing multi-modal fake information detection methods pay more attention to the formation of multi-modal features.The research on the contribution rate of cross-modal ambiguity and different modal features in detection is not perfect,ignoring the impact of inherent differences among different modal features on fake information detection.To solve the problem,this paper proposes to construct an improved fake information detection model based on cross-modal correlation ambiguity learning.Through cross-modal ambiguity learning of text and image features,the weights of unimodal features and fused features are updated by the ambiguity score.The unimodal features and fused features are combined adaptively,and the weights of text and image features are dynamically assigned by grid search to improve the detection accuracy.The effectiveness of the model is verified by experiments on the Twitter dataset.The accuracy is improved by 6% compared with the baseline model and 1.6% compared with the detection without dynamic weight assignment.

Key words: Fake news detection, Multimodal, Cross-modal correlation, Ambiguity learning, Fusion features

CLC Number: 

  • TP391
[1]JIN Z,CAO J,WANG B,et al.Research on social multimedia rumor detection technology integrating multi-modal features [J].Journal of Nanjing University of Information Science and Technology(Natural Science Edition),2017,9(6):583-592.
[2]ISLAM S,SARKAR T,KHAN S H,et al.COVID-19-related infodemic and its impact on public health:A global social media analysis[J].Am.J.Trop.Med.Hyg,2020,103(4),1621-1629.
[3]CAO J,SHENG Q,QI P.Progress and prospect of Internet false information detection [J].Communication of China Computer Society,2020,16(3):52-57.
[4]KHATTAR D,GOND J,GUPTA M,et al.MVAE:Multimodal Variational Autoencoder for Fake News Detection[C]//The World Wide Web Conference.2019:2915-2921.
[5]WANG Y Q,MA F L,JIN Z W,et al.EANN:Event Adversa-rial Neural Networks for Multi-Modal Fake News Detection[C]//ACM SIGKDD International Conference on Knowledge Discovery Data Mining(KDD).2018:849-857.
[6]POPAT K,MUKHERJEE S,YATES A,et al.DeClarE:Debunking Fake News and False Claims Using Evidence-Aware Deep Learning[C]//Proceeding of the 2018 Conference on Empirical Methods in Natural Language Processing(EMNLP),Brussels,Belgium.USA:ACL.2018:22-32.
[7]SONG C,SHU K,WU B.Temporally evolving graph neural network for fake news detection[J].Information Processing & Management,2021,58:102712.
[8]LIU Y,WU Y F B.Early detection of fake news on social media through propagation path classifica-tion with recurrent and convolutional networks[C]//The 32th AAAI Conference on Artificial Intelligence(AAAI-18).2018:354-361.
[9]XUE J,WANG Y,TIAN Y,et al.Detecting fake news by exploring the consistency of multimodal data[J].Information Processing & Management.2021,58(5):102610.
[10]ZHOU X,WU J,ZAFARANI R.Safe:Similarity-aware multi-modal fake news detection[J].arXiv:2003.04981,2020.
[11]ZHANG G,LI J.Detecting Social Media Fake News with Semantic Consistency Between Multi-model Contents[J].Data Analysis and Knowledge Discovery,2021,5(5):21-29.
[12]SHIVANGI S,MUDIT D,RAJIV R S,et al.Inter-Modality Discordance for Multimodal Fake News Detection[C]//ACM Multimedia Asia(MMAsia'21).2021:1-7.
[13]QI P,CAO J,LI X R,et al.Improving Fake News Detection by Using an Entity-enhanced Framework to Fuse Diverse Multi-modal Clues[C]//ACM MM21.2021:1212-1220.
[14]QI P,CAO J,SHENG Q.Semantics-Enhanced Multi-ModalFake News Detection[J].Journal of Computer Research and Development,2021,58(7):1456-1465.
[15]CHEN Y,LI D,ZHANG P,et al.Cross-modal ambiguity lear-ning for multimodal fake news detection[C]//Proceedings of the ACM Web Conference.2022:2897-2905.
[16]HUA J,CUI X D,LI X H,et al.Multimodal fake news detection through data augmentation-based contrastive learning[J].Applied Soft Computing,2023,136(C):1568-4946.
[17]YING Q,HU X,ZHOU Y,et al.Bootstrapping multi-view representations for fake news detection[C]//AAAI.2023.
[18]BOIDIDOU C,PAPADOPOULOS S,ZAM-POGLOU M,et al.Detection and visualization of misleading content on Twitter[J].International Journal of Multimedia Information Retrieval,2018,7(1):71-86.
[19]JIN Z,CAO J,GUO H,et al.Multimodal fusion with recurrent neural networks for rumor detection on microblogs[C]//Proceedings of the 25th ACM International Conference on Multimedia.2017:795-816.
[20]WU Y,ZHAN P,ZHANG Y,et al.Multimodal Fusion with Co-Attention Networks for Fake News Detection[OL].https://aclanthology.org/2021.findings-acl.226/.
[21]CHEN Y X,LI D S,ZHANG P,et al.Cross-modal Ambiguity Learning for Multimodal Fake News Detection[C]//Procee-dings of the ACM Web Conference 2022(WWW'22).Virtual Event,Lyon,France.ACM,New York,NY,USA,2022:2897-2905.
[1] TANG Jia, GUO Yan, YE Mingwei, WU Guixing. Multimodal Pre-training Method for Multi-view Contrastive Learning and Semantic Enhancement [J]. Computer Science, 2024, 51(1): 168-174.
[2] ZHANG Yian, YANG Ying, REN Gang, WANG Gang. Study on Multimodal Online Reviews Helpfulness Prediction Based on Attention Mechanism [J]. Computer Science, 2023, 50(8): 37-44.
[3] SONG Xinyang, YAN Zhiyuan, SUN Muyi, DAI Linlin, LI Qi, SUN Zhenan. Review of Talking Face Generation [J]. Computer Science, 2023, 50(8): 68-78.
[4] ZHOU Fengfan, LING Hefei, ZHANG Jinyuan, XIA Ziwei, SHI Yuxuan, LI Ping. Facial Physical Adversarial Example Performance Prediction Algorithm Based on Multi-modal Feature Fusion [J]. Computer Science, 2023, 50(8): 280-285.
[5] QIN Jing, WANG Weibin, ZOU Qijie, WANG Zumin, JI Changqing. Review of 3D Target Detection Methods Based on LiDAR Point Clouds [J]. Computer Science, 2023, 50(6A): 220400214-7.
[6] ZHANG Renbin, ZUO Yicong, ZHOU Zelin, WANG Long, CUI Yuhang. Multimodal Generative Adversarial Networks Based Multivariate Time Series Anomaly Detection [J]. Computer Science, 2023, 50(5): 355-362.
[7] CHEN Zhen, PU Yuanyuan, ZHAO Zhengpeng, XU Dan, QIAN Wenhua. Multimodal Sentiment Analysis Based on Adaptive Gated Information Fusion [J]. Computer Science, 2023, 50(3): 298-306.
[8] FAN Dongxu, GUO Yi. Aspect-based Multimodal Sentiment Analysis Based on Trusted Fine-grained Alignment [J]. Computer Science, 2023, 50(12): 246-254.
[9] LI Xiaopeng, LING Cheng, GAO Jingyang. Mixed Path HMC Sampling Methods for Molecular Tree Spaces [J]. Computer Science, 2023, 50(12): 322-329.
[10] ZHANG Lianfu, TAN Zuowen. Federated Learning Privacy-preserving Approach for Multimodal Medical Data [J]. Computer Science, 2023, 50(11A): 230800021-8.
[11] YANG Xiaomeng, ZHANG Tao, ZHUANG Jianjun, QIAO Xiaoqiang, DU Yihang. Modulation Signal Recognition Based on Multimodal Fusion and Deep Learning [J]. Computer Science, 2023, 50(11A): 220900007-7.
[12] NIE Xiu-shan, PAN Jia-nan, TAN Zhi-fang, LIU Xin-fang, GUO Jie, YIN Yi-long. Overview of Natural Language Video Localization [J]. Computer Science, 2022, 49(9): 111-122.
[13] CHANG Bing-guo, SHI Hua-long, CHANG Yu-xin. Multi Model Algorithm for Intelligent Diagnosis of Melanoma Based on Deep Learning [J]. Computer Science, 2022, 49(6A): 22-26.
[14] LI Hao-dong, HU Jie, FAN Qin-qin. Multimodal Multi-objective Optimization Based on Parallel Zoning Search and Its Application [J]. Computer Science, 2022, 49(5): 212-220.
[15] ZHAO Liang, ZHANG Jie, CHEN Zhi-kui. Adaptive Multimodal Robust Feature Learning Based on Dual Graph-regularization [J]. Computer Science, 2022, 49(4): 124-133.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!