计算机科学

计算机科学 ›› 2026, Vol. 53 ›› Issue (9): 324-332.doi: 10.11896/jsjkx.250800074

• 人工智能 • 上一篇    下一篇

ITRHG:基于超图的“图文合理化”虚假新闻检测方法

乔克祥1, 李旸2, 王素格1,3   

  1. 1 山西大学计算机与信息技术学院 太原 030006
    2 山西财经大学金融学院 太原 030006
    3 山西大学计算智能与中文信息处理教育部重点实验室 太原 030006
  • 收稿日期:2025-08-18 修回日期:2025-11-28 出版日期:2026-09-15 发布日期:2026-09-10
  • 通讯作者: 李旸(liyang@sxufe.edu.cn)
  • 作者简介:(qiaokexiang@sxu.edu.cn)
  • 基金资助:
    国家自然科学基金(62376143,U24A20335)

ITRHG:“Image-Text Rationalization” Fake News Detection Method Based on Hypergraph

QIAO Kexiang1, LI Yang2, WANG Suge1,3   

  1. 1 School of Computer and Information Technology,Shanxi University,Taiyuan 030006,China
    2 School of Finance,Shanxi University of Finance and Economics,Taiyuan 030006,China
    3 Key Laboratory Computational Intelligence and Chinese Information Processing of Ministry of Education,Shanxi University,Taiyuan 030006,China
  • Received:2025-08-18 Revised:2025-11-28 Published:2026-09-15 Online:2026-09-10
  • About author:QIAO Kexiang,born in 1999,postgra-duate.His main research interest is na-tural language processing.
    LI Yang,born in 1988,Ph.D,associate professor,is a member of CCF(No.P6278M).Her main research interests include text sentiment analysis and text mining.
  • Supported by:
    National Natural Science Foundation of China(62376143,U24A20335).

PDF (PC)

297

摘要： 虚假新闻通常选择与文本语义表面匹配但深层信息矛盾的图片,使新闻内容看似合理化,进而误导受众。因此,多模态虚假新闻检测面临“图文合理化”现象的挑战。对此,提出了一种基于超图的“图文合理化”虚假新闻检测方法。具体地,跨模态交互增强模块生成引导权重并进行双向Hadamard乘积,建立单模态特征的语义对齐;超图学习模块将单模态特征进行拼接,建立多模态新闻的表征,设计新闻节点与超边的连接强度度量及自适应超边数量调整策略,建立新闻间关系的超图,自动学习新闻之间复杂的高阶关系;多模态分类模块聚合单模态表征和高阶关系表征,用于多模态虚假新闻检测。实验结果表明,所提方法在多模态虚假新闻检测任务上取得了先进的结果,在中文Weibo数据集和英文Twitter数据集上,其准确率分别提升了1.6个百分点和3.3个百分点,F1值分别提升了1.6个百分点和3.8个百分点。

关键词: 多模态虚假新闻检测, 超图学习, 图文合理化, 高阶关系建模

Abstract: Fake news often uses images that superficially match the textual content but contradict it in deeper meaning,making the news appear plausible and misleading the audience.Therefore,multimodal fake news detection is confronted with the challenge of the “image-text rationalization” phenomenon.This paper proposes a “image-text rationalization” fake news detection based on hypergraph.The cross-modal interaction enhancement module generates guiding weights for bidirectional feature enhancement to establish semantic alignment of unimodal features.The hypergraph learning module concatenates unimodal features to form multimodal news representations,introduces a connection strength measure between news nodes and hyperedges,and applies an adaptive hyperedge number adjustment strategy to construct a hypergraph that captures complex high-order relations among news.The multimodal classification module aggregates unimodal and high-order relational representations for detection.Experimental results show that the proposed method achieves state-of-the-art performance,with accuracy gains of 1.6 percentage points on the Chinese Weibo dataset and 3.3 percentage points on the English Twitter dataset,and with F1 score gains of 1.6 percentage points and 3.8 percentage points,respectively.

Key words: Multimodal fake news detection, Hypergraph learning, Image-text rationalization, Higher-order relationship modeling

中图分类号: 

  • TP391
[1] HUH M,LIU A,OWENS A,et al.Fighting fake news:imagesplice detection via learned self-consistency[C] //Proceedings of the European Conference on Computer Vision(ECCV).2018:101-117.
[2] ZUBIAGA A,AKER A,BONTCHEVA K,et al.Detection and resolution of rumours in social media:a survey[J].ACM Computing Surveys,2018,51(2):1-36.
[3] ZHANG M D,ZHOU X,WU X H,et al.Unified fake news detection based on semantic expansion and HDGCN[J].Computer Science,2024,51(4):299-306.
[4] YU Y X,JI K,GAO Y,et al.Multi-source heterogeneous data progressive fusion for fake news detection[J].Computer Science,2024,51(11):30-38.
[5] XIAO L,ZHANG Q,SHI C,et al.Msynfd:multi-hop syntax aware fake news detection[C] //Proceedings of the ACM Web Conference 2024.2024:4128-4137.
[6] ZHANG Y,MA X,WU J,et al.Heterogeneous subgraph transformer for fake news detection[C] //Proceedings of the ACM Web Conference.2024:1272-1282.
[7] YIN J,GAO M,SHU K,et al.Fine-grained discrepancy contrastive learning for robust fake news detection[C] //Proceedings of the ICASSP 2024－2024 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP).2024:12541-12545.
[8] HUANG Z,HU Y,ZENG Z,et al.Multimodal stacked cross attention network for fine-grained fake news detection[C] //Proceedings of the IEEE International Conference on Multimedia and Expo(ICME).2023:2837-2842.
[9] ZENG Z,WU M,LI G,et al.An explainable multi-view semantic fusion model for multimodal fake news detection[C] //Proceedings of the IEEE International Conference on Multimedia and Expo(ICME).2023:1235-1240.
[10] BAI Y,LIU Y,LI Y.Learning frequency-aware cross-modal interaction for multimodal fake news detection[J].IEEE Transactions on Computational Social Systems.2024,11(5):6568-6579.
[11] ZHANG L,ZHANG X,LI C,et al.Mitigating social hazards:early detection of fake news via diffusion-guided propagation path generation[C] //Proceedings of the 32nd ACM International Conference on Multimedia.2024:2842-2851.
[12] ZHANG Q,LIU J,ZHANG F,et al.Natural language-centered inference network for multi-modal fake news detection[C] //Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence.2024:2542-2550.
[13] ZHANG L,ZHANG X,ZHUO Z,et al.Reinforced adaptiveknowledge learning for multimodal fake news detection[C] //Proceedings of the AAAI Conference On Artificial Intelligence.2024:16777-16785.
[14] LIU S,YUE X,WU F,et al.Semantic distillation and structural alignment network for fake news detection[C] //Proceedings of the ICASSP 2024-2024 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP).2024:6620-6624.
[15] YI Z,LU S,TANG X,et al.Maccn:multi-modal adaptive co-attention fusion contrastive learning networks for fake news detection[C] //Proceedings of the ICASSP 2024-2024 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP).2024:6045-6049.
[16] WU F,JIN H,HU C,et al.Efficient cross-modal prompt lear-ning with semantic enhancement for domain-robust fake news detection[C] //Proceedings of the 31st International Conference on Computational Linguistics.2025:4175-4185.
[17] SHEN L,LONG Y,CAI X,et al.Gamed:knowledge adaptive multi-experts decoupling for multimodal fake news detection[C] //Proceedings of the Eighteenth ACM International Confe-rence on Web Search and Data Mining.2025:586-595.
[18] MAHMUD T,HASAN I,AZIZ M T,et al.Enhanced fake news detection through the fusion of deep learning and repeat vector representations[C] //Proceedings of the 2024 2nd International Conference on Intelligent Data Communication Technologies and Internet of Things(IDCIoT).2024:654-660.
[19] LIU H.Decoupled user preference enhanced fake news detection[C] //Proceedings of the 2024 3rd International Conference on Cyber Security,Artificial Intelligence and Digital Economy.2024:61-65.
[20] YIN S,ZHU P,WU L,et al.Gamc:an unsupervised method for fake news detection using graph autoencoder with masking[C] //Proceedings of the AAAI Conference On Artificial Intelligence.2024:347-355.
[21] WANG L,ZHANG C,XU H,et al.Cross-modal contrastivelearning for multimodal fake news detection[C] //Proceedings of the 31st ACM International Conference on Multimedia.2023:5696-5704.
[22] LIU G F,ZHANG J H,LIU Q,et al.Uni-modal event-agnostic knowledge distillation for multimodal fake news detection[J].IEEE Transactions on Knowledge and Data Engineering,2024,36(12):9490-9503.
[23] MA Z,LUO M,GUO H,et al.Event-radar:event-driven multi-view learning for multimodal fake news detection[C] //Procee-dings of the 62nd Annual Meeting of the Association for Computational Linguistics.2024:5809-5821.
[24] TONG Y,LU W,ZHAO Z,et al.Mmdfnd:multi-modal multi-domain fake news detection[C] //Proceedings of the 32nd ACM International Conference on Multimedia.2024:1178-1186.
[25] HU B D,WANG X G,WANG X Y,et al.Survey on hypergraph learning:algorithm classification and application analysis[J].Journal of Software.2021,33(2):498-523.
[26] CHENG D,CHEN J,PENG W,et al.Ihgnn:interactive hyper-graph neural network for personalized product search[C] //Proceedings of the ACM Web Conference.2022:256-265.
[27] JEONG U,DING K,CHENG L,et al.Nothing stands alone:relational fake news detection with hypergraph neural networks[C] //Proceedings of the 2022 IEEE International Conference on Big Data(Big Data).2022:596-605.
[28] SU X,YANG J,WU J,et al.Hy-defake:hypergraph neural networks for detecting fake news in online social networks[J].Neural Networks.2025,187:107302.
[29] CHENG Z,ZHANG J,XU X,et al.Retrieval-augmented hypergraph for multimodal social media popularity prediction[C] //Proceedings of the 30th ACM SIGKDD Conference on Know-ledge Discovery and Data Mining.2024:445-455.
[30] LIU J,LI T,WU D,et al.An aspect performance-aware hypergraph neural network for review-based recommendation[C] //Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining.2025:503-511.
[31] YE R,PEI X.Multi-view fake news detection model based ondynamic hypergraph[J].arXiv:2412.19227,2024.
[32] SALAMANOS N,LEONIDOU P,LAOUTARIS N,et al.Hy-pergraphdis:leveraging hypergraphs for contextual and social-based disinformation detection[C] //Proceedings of the International AAAI Conference on Web and Social Media.2024:1381-1394.
[33] CHENG L,ZHU P,GUO Y,et al.Hyperdet:source detection in hypergraphs via interactive relationship construction and feature-rich attention fusion[J].arXiv:2505.12894,2025.
[34] DEVLIN J,CHANG M W,LEE K,et al.Bert:pre-training of deep bidirectional transformers for language understanding[C] //Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2019:4171-4186.
[35] DOSOVITSKIY A,BEYER L,KOLESNIKOV A,et al.Animage is worth 16x16 words:transformers for image recognition at scale[J].arXiv:2010.11929,2020.
[36] ZHOU P,WU Z,ZENG X,et al.Totally dynamic hypergraphneural network[C] //Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence.2023:2476-2483.
[37] GUO H,LI X Y,TANG J Y,et al.Adaptive feature fusion for multi-mod-al entity alignment[J].Acta Automatica Sinica,2024,50(4):758-770.
[38] JIN Z,CAO J,GUO H,et al.Multimodal fusion with recurrent neural networks for rumor detection on microblogs[C] //Proceedings of the 25th ACM International Conference on Multimedia.2017:795-816.
[39] BOIDIDOU C,PAPADOPOULOSS,ZAMPOGLOU M,et al.Detection and visualization of misleading content on Twitter[J].International Journal of Multimedia Information Retrieval,2018,7(1):71-86.
[40] WANG Y,MA F,JIN Z,et al.Eann:event adversarial neuralnetworks for multi-modal fake news detection[C] //Proceedings of the 24th ACM SIGKDD International Conference on Know-ledge Discovery & Data Mining.2018:849-857.
[41] KHATTAR D,GOUD J S,GUPTA M,et al.Mvae:multimodal variational autoencoder for fake news detection[C] //The World Wide Web Conference.2019:2915-2921.
[42] WU Y,ZHAN P,ZHANG Y,et al.Multimodal fusion with co-attention networks for fake news detection[C] //Findings of the Association for Computational Linguistics:ACL-IJCNLP.2021:2560-2569.
[43] CHEN Y,LI D,ZHANG P,et al.Cross-modal ambiguity lear-ning for multimodal fake news detection[C] //Proceedings of the ACM Web Conference.2022:2897-2905.
[44] ZHANG Q,LIU J,ZHANG F,et al.Hierarchical semantic enhancement network for multimodal fake news detection[C] //Proceedings of the 31st ACM International Conference on Multimedia.2023:3424-3433.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市两江新区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发