Computer Science ›› 2026, Vol. 53 ›› Issue (9): 324-332.doi: 10.11896/jsjkx.250800074

• Artificial Intelligence • Previous Articles     Next Articles

ITRHG:“Image-Text Rationalization” Fake News Detection Method Based on Hypergraph

QIAO Kexiang1, LI Yang2, WANG Suge1,3   

  1. 1 School of Computer and Information Technology,Shanxi University,Taiyuan 030006,China
    2 School of Finance,Shanxi University of Finance and Economics,Taiyuan 030006,China
    3 Key Laboratory Computational Intelligence and Chinese Information Processing of Ministry of Education,Shanxi University,Taiyuan 030006,China
  • Received:2025-08-18 Revised:2025-11-28 Online:2026-09-15 Published:2026-09-10
  • About author:QIAO Kexiang,born in 1999,postgra-duate.His main research interest is na-tural language processing.
    LI Yang,born in 1988,Ph.D,associate professor,is a member of CCF(No.P6278M).Her main research interests include text sentiment analysis and text mining.
  • Supported by:
    National Natural Science Foundation of China(62376143,U24A20335).

Abstract: Fake news often uses images that superficially match the textual content but contradict it in deeper meaning,making the news appear plausible and misleading the audience.Therefore,multimodal fake news detection is confronted with the challenge of the “image-text rationalization” phenomenon.This paper proposes a “image-text rationalization” fake news detection based on hypergraph.The cross-modal interaction enhancement module generates guiding weights for bidirectional feature enhancement to establish semantic alignment of unimodal features.The hypergraph learning module concatenates unimodal features to form multimodal news representations,introduces a connection strength measure between news nodes and hyperedges,and applies an adaptive hyperedge number adjustment strategy to construct a hypergraph that captures complex high-order relations among news.The multimodal classification module aggregates unimodal and high-order relational representations for detection.Experimental results show that the proposed method achieves state-of-the-art performance,with accuracy gains of 1.6 percentage points on the Chinese Weibo dataset and 3.3 percentage points on the English Twitter dataset,and with F1 score gains of 1.6 percentage points and 3.8 percentage points,respectively.

Key words: Multimodal fake news detection, Hypergraph learning, Image-text rationalization, Higher-order relationship modeling

CLC Number: 

  • TP391
[1] HUH M,LIU A,OWENS A,et al.Fighting fake news:imagesplice detection via learned self-consistency[C] //Proceedings of the European Conference on Computer Vision(ECCV).2018:101-117.
[2] ZUBIAGA A,AKER A,BONTCHEVA K,et al.Detection and resolution of rumours in social media:a survey[J].ACM Computing Surveys,2018,51(2):1-36.
[3] ZHANG M D,ZHOU X,WU X H,et al.Unified fake news detection based on semantic expansion and HDGCN[J].Computer Science,2024,51(4):299-306.
[4] YU Y X,JI K,GAO Y,et al.Multi-source heterogeneous data progressive fusion for fake news detection[J].Computer Science,2024,51(11):30-38.
[5] XIAO L,ZHANG Q,SHI C,et al.Msynfd:multi-hop syntax aware fake news detection[C] //Proceedings of the ACM Web Conference 2024.2024:4128-4137.
[6] ZHANG Y,MA X,WU J,et al.Heterogeneous subgraph transformer for fake news detection[C] //Proceedings of the ACM Web Conference.2024:1272-1282.
[7] YIN J,GAO M,SHU K,et al.Fine-grained discrepancy contrastive learning for robust fake news detection[C] //Proceedings of the ICASSP 2024-2024 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP).2024:12541-12545.
[8] HUANG Z,HU Y,ZENG Z,et al.Multimodal stacked cross attention network for fine-grained fake news detection[C] //Proceedings of the IEEE International Conference on Multimedia and Expo(ICME).2023:2837-2842.
[9] ZENG Z,WU M,LI G,et al.An explainable multi-view semantic fusion model for multimodal fake news detection[C] //Proceedings of the IEEE International Conference on Multimedia and Expo(ICME).2023:1235-1240.
[10] BAI Y,LIU Y,LI Y.Learning frequency-aware cross-modal interaction for multimodal fake news detection[J].IEEE Transactions on Computational Social Systems.2024,11(5):6568-6579.
[11] ZHANG L,ZHANG X,LI C,et al.Mitigating social hazards:early detection of fake news via diffusion-guided propagation path generation[C] //Proceedings of the 32nd ACM International Conference on Multimedia.2024:2842-2851.
[12] ZHANG Q,LIU J,ZHANG F,et al.Natural language-centered inference network for multi-modal fake news detection[C] //Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence.2024:2542-2550.
[13] ZHANG L,ZHANG X,ZHUO Z,et al.Reinforced adaptiveknowledge learning for multimodal fake news detection[C] //Proceedings of the AAAI Conference On Artificial Intelligence.2024:16777-16785.
[14] LIU S,YUE X,WU F,et al.Semantic distillation and structural alignment network for fake news detection[C] //Proceedings of the ICASSP 2024-2024 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP).2024:6620-6624.
[15] YI Z,LU S,TANG X,et al.Maccn:multi-modal adaptive co-attention fusion contrastive learning networks for fake news detection[C] //Proceedings of the ICASSP 2024-2024 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP).2024:6045-6049.
[16] WU F,JIN H,HU C,et al.Efficient cross-modal prompt lear-ning with semantic enhancement for domain-robust fake news detection[C] //Proceedings of the 31st International Conference on Computational Linguistics.2025:4175-4185.
[17] SHEN L,LONG Y,CAI X,et al.Gamed:knowledge adaptive multi-experts decoupling for multimodal fake news detection[C] //Proceedings of the Eighteenth ACM International Confe-rence on Web Search and Data Mining.2025:586-595.
[18] MAHMUD T,HASAN I,AZIZ M T,et al.Enhanced fake news detection through the fusion of deep learning and repeat vector representations[C] //Proceedings of the 2024 2nd International Conference on Intelligent Data Communication Technologies and Internet of Things(IDCIoT).2024:654-660.
[19] LIU H.Decoupled user preference enhanced fake news detection[C] //Proceedings of the 2024 3rd International Conference on Cyber Security,Artificial Intelligence and Digital Economy.2024:61-65.
[20] YIN S,ZHU P,WU L,et al.Gamc:an unsupervised method for fake news detection using graph autoencoder with masking[C] //Proceedings of the AAAI Conference On Artificial Intelligence.2024:347-355.
[21] WANG L,ZHANG C,XU H,et al.Cross-modal contrastivelearning for multimodal fake news detection[C] //Proceedings of the 31st ACM International Conference on Multimedia.2023:5696-5704.
[22] LIU G F,ZHANG J H,LIU Q,et al.Uni-modal event-agnostic knowledge distillation for multimodal fake news detection[J].IEEE Transactions on Knowledge and Data Engineering,2024,36(12):9490-9503.
[23] MA Z,LUO M,GUO H,et al.Event-radar:event-driven multi-view learning for multimodal fake news detection[C] //Procee-dings of the 62nd Annual Meeting of the Association for Computational Linguistics.2024:5809-5821.
[24] TONG Y,LU W,ZHAO Z,et al.Mmdfnd:multi-modal multi-domain fake news detection[C] //Proceedings of the 32nd ACM International Conference on Multimedia.2024:1178-1186.
[25] HU B D,WANG X G,WANG X Y,et al.Survey on hypergraph learning:algorithm classification and application analysis[J].Journal of Software.2021,33(2):498-523.
[26] CHENG D,CHEN J,PENG W,et al.Ihgnn:interactive hyper-graph neural network for personalized product search[C] //Proceedings of the ACM Web Conference.2022:256-265.
[27] JEONG U,DING K,CHENG L,et al.Nothing stands alone:relational fake news detection with hypergraph neural networks[C] //Proceedings of the 2022 IEEE International Conference on Big Data(Big Data).2022:596-605.
[28] SU X,YANG J,WU J,et al.Hy-defake:hypergraph neural networks for detecting fake news in online social networks[J].Neural Networks.2025,187:107302.
[29] CHENG Z,ZHANG J,XU X,et al.Retrieval-augmented hypergraph for multimodal social media popularity prediction[C] //Proceedings of the 30th ACM SIGKDD Conference on Know-ledge Discovery and Data Mining.2024:445-455.
[30] LIU J,LI T,WU D,et al.An aspect performance-aware hypergraph neural network for review-based recommendation[C] //Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining.2025:503-511.
[31] YE R,PEI X.Multi-view fake news detection model based ondynamic hypergraph[J].arXiv:2412.19227,2024.
[32] SALAMANOS N,LEONIDOU P,LAOUTARIS N,et al.Hy-pergraphdis:leveraging hypergraphs for contextual and social-based disinformation detection[C] //Proceedings of the International AAAI Conference on Web and Social Media.2024:1381-1394.
[33] CHENG L,ZHU P,GUO Y,et al.Hyperdet:source detection in hypergraphs via interactive relationship construction and feature-rich attention fusion[J].arXiv:2505.12894,2025.
[34] DEVLIN J,CHANG M W,LEE K,et al.Bert:pre-training of deep bidirectional transformers for language understanding[C] //Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2019:4171-4186.
[35] DOSOVITSKIY A,BEYER L,KOLESNIKOV A,et al.Animage is worth 16x16 words:transformers for image recognition at scale[J].arXiv:2010.11929,2020.
[36] ZHOU P,WU Z,ZENG X,et al.Totally dynamic hypergraphneural network[C] //Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence.2023:2476-2483.
[37] GUO H,LI X Y,TANG J Y,et al.Adaptive feature fusion for multi-mod-al entity alignment[J].Acta Automatica Sinica,2024,50(4):758-770.
[38] JIN Z,CAO J,GUO H,et al.Multimodal fusion with recurrent neural networks for rumor detection on microblogs[C] //Proceedings of the 25th ACM International Conference on Multimedia.2017:795-816.
[39] BOIDIDOU C,PAPADOPOULOSS,ZAMPOGLOU M,et al.Detection and visualization of misleading content on Twitter[J].International Journal of Multimedia Information Retrieval,2018,7(1):71-86.
[40] WANG Y,MA F,JIN Z,et al.Eann:event adversarial neuralnetworks for multi-modal fake news detection[C] //Proceedings of the 24th ACM SIGKDD International Conference on Know-ledge Discovery & Data Mining.2018:849-857.
[41] KHATTAR D,GOUD J S,GUPTA M,et al.Mvae:multimodal variational autoencoder for fake news detection[C] //The World Wide Web Conference.2019:2915-2921.
[42] WU Y,ZHAN P,ZHANG Y,et al.Multimodal fusion with co-attention networks for fake news detection[C] //Findings of the Association for Computational Linguistics:ACL-IJCNLP.2021:2560-2569.
[43] CHEN Y,LI D,ZHANG P,et al.Cross-modal ambiguity lear-ning for multimodal fake news detection[C] //Proceedings of the ACM Web Conference.2022:2897-2905.
[44] ZHANG Q,LIU J,ZHANG F,et al.Hierarchical semantic enhancement network for multimodal fake news detection[C] //Proceedings of the 31st ACM International Conference on Multimedia.2023:3424-3433.
[1] MENG Xiangfu, WANG Wanchun, ZHANG Yumeng, FAN Wenyi. Multiple Attention Mechanism News Recommendation Approach with Hypergraph Learning [J]. Computer Science, 2025, 52(11A): 250200067-7.
[2] YE Jun and JIN Zhong. Hypergraph Dual Regularization Concept Factorization Algorithm and Its Application in Data Representation [J]. Computer Science, 2017, 44(7): 309-314.
[3] HONG Chao-qun, CHEN Xu-hui, WANG Xiao-dong, LI Shi-jin and WU Ke-shou. Hypergraph Dimensionality Reduction with Multiple Feature Fusion Based on GPU Parallel Acceleration [J]. Computer Science, 2015, 42(11): 90-93.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!