计算机科学

计算机科学 ›› 2025, Vol. 52 ›› Issue (12): 391-399.doi: 10.11896/jsjkx.241000161

• 信息安全 • 上一篇    下一篇

基于主动学习的多模态谣言检测模型

商云娴, 蔡国永, 刘庆华, 蒋艺明   

  1. 桂林电子科技大学计算机与信息安全学院 广西 桂林 541004
    广西可信软件重点实验室 广西 桂林 541004
  • 收稿日期:2024-10-28 修回日期:2025-03-03 出版日期:2025-12-15 发布日期:2025-12-09
  • 通讯作者: 蔡国永(ccgycai@guet.edu.cn)
  • 作者简介:(22032202027@mails.guet.edu.cn)
  • 基金资助:
    国家自然科学基金(62366010);广西自然科学基金(2024GXNSFAA010374)

Active Learning-based Multi-modal Fusion Rumor Detection

SHANG Yunxian, CAI Guoyong, LIU Qinghua, JIANG Yiming   

  1. College of Computer and Information Security, Guilin University of Electronic Technology, Guilin, Guangxi 541004, ChinaKey Laboratory of Guangxi Trusted Software, Guilin, Guangxi 541004, China
  • Received:2024-10-28 Revised:2025-03-03 Published:2025-12-15 Online:2025-12-09
  • About author:SHANG Yunxian,born in 2000,postgraduate,is a member of CCF(No.U6275G).Her main research interests include natural language processing and rumor detection.
    CAI Guoyong,born in 1971,Ph.D,professor,Ph.D supervisor,is a distinguished member of CCF(No.12524D).His main research interests include multi-modal affective computing,trus-table AI theory and techniques.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China(62366010) and Guangxi Natural Science Foundation(2024GXNSFAA010374).

PDF (PC)

4

摘要： 传统的谣言检测方法仍然有许多不足,如没有充分利用多模态信息,未考虑样本稀缺、标注昂贵、领域变化等实际情况,无法满足需求。为了解决样本稀缺和领域变化的问题,提出了一种新的基于主动学习的多模态谣言检测模型ALMF。ALMF设计了一种基于传播结构图增强的不确定性查询策略,使得主动学习筛选的样本更具有学习价值,同时减少了样本标注量;另外,ALMF使用多模态数据,充分结合文本特征、图像特征以及传播结构特征,通过不同模态特征之间的互补增强,提升了谣言检测能力。在PHEME和WEIBO两个数据集上进行了实验,结果表明,ALMF的性能优于对比模型,准确率提高2%~9%。相比于基于基础查询策略的主动学习,ALMF仅标注约5%的样本取得的性能与全样本时的性能相当。通过使用传播结构图增强的查询策略和跨模态增强的融合方式,ALMF模型成功应对了新领域事件谣言检测面临的挑战。

关键词: 谣言检测, 主动学习, 多模态融合, 协同注意力, 传播图增强

Abstract: Traditional rumor detection methods still have many shortcomings,such as insufficient utilization of multi-modal information,failure to consider sample scarcity,high labeling costs,and domain shifts.Therefore,it cannot meet the demands.To address the issues of sample scarcity and domain changes,this paper proposes a new Active Learning-based Multi-modal Fusion Rumor detection model,called ALMF.ALMF designs a novel uncertainty query strategy enhanced by the propagation structure graph,which ensures that the samples selected through active learning have greater learning value and reduces the demand for sample labeling.Meanwhile,ALMF employs multi-modal data,fully integrating text features,image features,and propagation structure features.The complementary enhancement between different modal features improves the capability of rumor detection.ALMF is tested on the PHEME and WEIBO datasets.The results show that ALMF outperforms the compared models,achieving an accuracy improvement of 2% to 9%.Compared to active learning based on basic query strategies,ALMF achieves performance that is nearly equivalent to that of full sample utilization with only approximately 5% of the samples labeled.By employing a query strategy enhanced with propagation structure graphs and cross-modal enhancement fusion methods,the ALMF model successfully addresses the challenges associated with rumor detection in new domain events.

Key words: Rumor detection, Active learning, Multi-modal fusion, Co-attention, Propagation graph enhancement

中图分类号: 

  • TP391
[1]GAO Y,LIANG G,JIANG F,et al.Overview of social network rumor detection [J].Acta Electronica Sinica,2020,48(7):1421.
[2]SHU K,SLIVA A,WANG S,et al.Fake news detection on social media:A data mining perspective[J].ACM SIGKDD Explorations Newsletter,2017,19(1):22-36.
[3]LIU H,CHEN S,CAO S,et al.Research on False News Detection based on multi-modal Learning [J].Exploration of Compu-ter Science and Technology,2023,17(9):2015-2029.
[4]JIN Z,CAO J,ZHANG Y,et al.Novel visual and statisticalimage features for microblogs news verification[J].IEEE Transactions on Multimedia,2016,19(3):598-608.
[5]JIN Z,CAO J,HAN G,et al.Multimodal Fusion with Recurrent Neural Networks for Rumor Detection on Microblogs[C]// Proceedings of the 25th ACM International Conference on Multimedia.ACM,2017:795-816.
[6]LU H Y,FAN C Y,WU X J.Small-sample COVID-19 rumor detection for network social media [J].Journal of Chinese Information Technology,2022,36(1):135-144,172.
[7]RAN H,JIA C.Unsupervised cross-domain rumor detectionwith contrastive learning and cross-attention[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2023:13510-13518.
[8]LIN H,YI P,MA J,et al.Zero-shot rumor detection with propagation structure via prompt learning[C]// Proceedings of the AAAI Conference on Artificial Intelligence.2023:5213-5221.
[9]HASAN M S,ALAM R,ADNAN M A.Truth or lie:Pre-emptive detection of fake news in different languages through entropy-based active learning and multi-model neural ensemble[C]//2020 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining(ASONAM).IEEE,2020:55-59.
[10]CASTILLO C,MENDOZA M,POBLETE B.Information credibility on twitter[C]//Proceedings of the 20th International Conference on World Wide Web.2011:675-684.
[11]YANG F,LIU Y,YU X,et al.Automatic detection of rumor on sina weibo[C]//Proceedings of the ACM SIGKDD Workshop on Mining Data Semantics.2012:1-7.
[12]YANG Y,ZHENG L,ZHANG J,et al.TI-CNN:Convolutional Neural Networks for Fake News Detection[J].arXiv:1806.00749,2018.
[13]YANG X,LYU Y,TIAN T,et al.Rumor detection on socialmedia with graph structured adversarial learning[C]//Procee-dings of the Twenty-ninth International Conference on International Joint Conferences on Artificial Intelligence.2021:1417-1423.
[14]WEN W,SU S,YU Z.Cross-lingual cross-platform rumor verification pivoting on multimedia content[J].arXiv:1808.04911,2018.
[15]ZUO Y,ZHU W,CAI G G.Continually detection,rapidly react:unseen rumors detection based on continual prompt-tuning[C]//Proceedings of the 29th International Conference on Computational Linguistics.2022:3029-3041.
[16]SETTLES B.Active learning literature survey:TR1648 [R].University of Wisconsin-Madison Department of Computer Sciences,2009.
[17]SAHAN M,SMIDL V,MARIK R.Active learning for text classification and fake news detection[C]//2021 International Symposium on Computer Science and Intelligent Controls(ISCSIC).IEEE,2021:87-94.
[18]FARINNEYA P,POUR M M A,HAMIDIAN S,et al.Active learning for rumor identification on social media[C]//Findings of the Association for Computational Linguistics:EMNLP 2021.2021:4556-4565.
[19]SHAO Z,CAI G,LIU Q,et al.An Active Learning Framework for Continuous Rapid Rumor Detection in Evolving Social Media[C]//2024 International Joint Conference on Neural Networks(IJCNN).IEEE,2024:1-8.
[20]CUI C,JIA C.Propagation Tree Is Not Deep:Adaptive Graph Contrastive Learning Approach for Rumor Detection[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2024:73-81.
[21]TANG Y P,LI G X,HUANG S J.ALiPy:Active learning in python[J].arXiv:1901.03802,2019.
[22]HE K,ZHANG X,REN S,et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[23]ZHENG J,ZHANG X,GUO S,et al.MFAN:Multi-modal Feature-enhanced Attention Networks for Rumor Detection[C]//IJCAI.2022:2413-2419.
[24]SONG C,YANG C,CHEN H,et al.CED:Credible early detection of social media rumors[J].IEEE Transactions on Know-ledge and Data Engineering,2019,33(8):3035-3047.
[25]ZUBIAGA A,LIAKATA M,PROCTER R.Exploiting context for rumour detection in social media[C]//Social Informatics:9th International Conference.Springer,2017:109-123.
[26]SUN M,LI J,GUO Z,et al.Thuctc:an efficient chinese textclassifier[R].GitHub Repository,2016.
[27]WANG Y,MA F,JIN Z,et al.Eann:Event adversarial neural networks for multi-modal fake news detection[C]//Proceedings of the 24th ACM SIGKDD International Conference on Know-ledge Discovery & Data Mining.2018:849-857.
[28]KHATTAR D,GOUD J S,GUPTA M,et al.Mvae:Multimodal variational autoencoder for fake news detection[C]//The world Wide Web Conference.2019:2915-2921.
[29]YUAN C,MA Q,ZHOU W,et al.Jointly embedding the local and global relations of heterogeneous graph for rumor detection[C]//2019 IEEE International Conference on Data Mining(ICDM).IEEE,2019:796-805.
[30]CHEN F,LI X,LI Z,et al.Multimodal Rumor Detection viaMultimodal Prompt Learning[C]//2024 International Joint Conference on Neural Networks(IJCNN).IEEE,2024:1-8.
[31]LIU T,CAI Q,XU C,et al.Rumor Detection with a novel graph neural network approach[J].arXiv:2403.16206,2024.
[32]YI F,LIU H,HE H,et al.A Comparative Analysis of Active Learning for Rumor Detection on Social Media Platforms[J].Applied Sciences,2023,13(22):12098.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发