Computer Science ›› 2026, Vol. 53 ›› Issue (8): 257-265.doi: 10.11896/jsjkx.250700054

• Artificial Intelligence • Previous Articles     Next Articles

Fake News Detection Model Based on Cross-modal Feature Fusion and Alignment

YANG Chenguang, LU Jicang, GUO Jiaxing   

  1. Information Engineering University, Zhengzhou 450001, China
  • Received:2025-07-10 Revised:2025-11-15 Online:2026-08-15 Published:2026-08-17
  • About author:YANG Chenguang,born in 1996,postgraduate.His main research interests include fake news detection and big data analysis.
    LU Jicang,born in 1985,Ph.D,associate professor.His main research interests include knowledge reasoning and social network analysis.

Abstract: Fake news is usually spread by exaggerated,distorted or misleading statements,which can shape negative social opinion and seriously endanger public safety.The current fake news is usually multi-modal,and the existing detection methods often fuse the features of each modality after extracting them separately,ignoring the correlation between modalities,which makes it difficult to fully capture the details and correlation information,resulting in unsatisfactory detection performance.To solve these pro-blems,this paper proposes a detection model based on cross-modal feature fusion and alignment(CMFFA).CMFFA optimizes the modes of feature extraction,fusion and classification,extracts modal features from macro and micro perspectives,enhances features through the attention mechanism,and evaluates the ambiguity between modalities by calculating the similarity between modalities,so as to adaptively perform cross-modal feature fusion.Firstly,the pre-trained model is used to encode the single-modal features and cross-modal features of text and image.Then,in the cross-modal feature fusion,the proportion of cross-modal features is adaptively adjusted through the ambiguity analysis between modalities,so as to better realize the fusion of cross-modal features and improve the performance of fake news detection.Experimental results show that,compared with the existing fake news detection methods on the public Chinese and English fake news datasets,the proposed model significantly improves the F1 value,precision and recall rate,verifying its effectiveness and superiority.

Key words: Fake news detection, Cross-modal feature fusion, Inter-modal ambiguity analysis, Attention mechanism

CLC Number: 

  • TP391
[1] HU Z R,HUANG C X.China New Media Development Report No.15[M]//Beijing:Social Sciences Academic Press,2024.
[2] BONDIELLI A,MARCELLONI F.A Survey on Fake News and Rumor Detection Techniques[J].Information Sciences,2019,497:38-55.
[3] KUMAR K,GEETHAKUMARI G.Detecting Misinformationin Online Social Networks Using Cognitive Psychology[J].Human-centric Computing and Information Sciences,2014,4(1):14-26.
[4] TRUEMAN T E,KUMAR A,NARAYANASAMY P,et al.Attention-based C-BiLSTM for Fake News Detection[J].Applied Soft Computing,2021,110:107600.
[5] WANG W Y.“Liar,Liar Pants on Fire”:a New Benchmark Dataset for Fake News Detection[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Stroudsburg:ACL,2017:422-426.
[6] CHAWDA S,PATIL A,SINGH A,et al.A Novel Approach for Clickbait Detection[C]//Proceedings of the 3rd International Conference on Trends in Electronics and Informatics.2019:1318-1321.
[7] RAI N,KUMAR D,KAUSHIK N,et al.Fake news classification using transformer based enhanced LSTM and BERT[C]//International Journal of Cognitive Computing in Engineering.2022:98-105.
[8] AHN Y,JEONG C.Natural Language Contents Evaluation System for Detecting Fake News Using Deep Learning[C]//Proceedings of the 16th International Joint Conference on Computer Science and Software Engineering.IEEE,2019:289-292.
[9] DEVLIN J,CHANG M W,LEE K,et al.BERT:Pre-training of Deep Bidirectional Transformers for Language Understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologie.2019:4171-4186.
[10] YU F,LIU Q,WU S,et al.A Convolutional Approach for Misinformation Identification[C]//Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence.2017:3901-3907.
[11] RAZA N,ABDULKADIR S J,ABID Y A,et al.Enhancing fake news detection with transformer-based deep learning:A multidisciplinary approach[J].PLoS One,2025,20(9):e0330954.
[12] XU F,LI M H,HUANG Q,et al.Rumor Detection Model based on Graph Convolutional Neural Network Driven by Knowledge Graph[J].Science China Information Science,2023,53(4):663-681.
[13] LIANG X,ZHANG Q,SHI C,et al.MSynFD:Multi-hop Syntax Aware Fake News Detection[C]//Proceedings of the ACM on Web Conference.2024:4128-4137.
[14] JIN Z W,CAO J,GUO H,et al.Multimodal Fusion with Recurrent Neural Networks for Rumor Detection on Microblogs[C]//Proceedings of the 25th ACM International Conference on Multimedia.2017:795-816.
[15] WANG Y,MA F,JIN Z,et al.EANN:Event Adversarial Neural Networks for Multi-Modal Fake News Detection[C]//Procee-dings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.2018:849-857.
[16] KIM Y.Convolutional Neural Networks for Sentence Classification[J].arXiv:1408.5882,2014.
[17] SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-Scale Image Recognition[J].arXiv:1409.1556,2014.
[18] ZHANG H,FANG Q,QIAN S,et al.Multi-modal Knowledge-Aware Event Memory Network for Social Media Rumor Detection[C]//Proceedings of the 27th ACM Conference on Multimedia.2019:1942-1951.
[19] WU Y,ZHAN P,ZHANG Y,et al.Multimodal Fusion with Co-Attention Networks for Fake News Detection[C]//Proceedings of the Findings Association for Computational Linguistics.2021:2560-2569.
[20] QIAN S,WANG J,HU J,et al.Hierarchical Multi-Modal Contextual Attention Network for Fake News Detection[C]//Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval.2021:153-162.
[21] SONG C,NING N W,ZHANG Y L,et al.A Multimodal Fake News Detection Model Based on Crossmodal Attention Residual And Multichannel Convolutional Neural Networks[J].Information Processing & Management,2021,58(1):102437.
[22] HE K,ZHANG X,REN S,et al.Deep Residual Learning forImage Recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[23] LIU Y,WU Y F.Early Detection of Fake News on Social Media Through Propagation Path Classification with Recurrent And Convolutional Networks[C]//Proceedings of the AAAI Confe-rence on Artificial Intelligence.2018:354-361.
[24] JING J,WU H,SUN J,et al.Multimodal Fake News Detection via Progressive Fusion Networks[J].Information Processing & Management,2023,60(1):103120.
[25] LIU Z,LIN Y,CAO Y,et al.Swin Transformer:Hierarchical Vision Transformer using Shifted Windows[C]//Proceedings of IEEE/CVF International Conference on Computer Vision.2021:9992-10002.
[26] ZHOU X Y,WU J D,ZAFARANI R.SAFE:Similarity-Aware Multi-modal Fake News Detection[C]//Proceedings of Pacific-Asia Conference on Knowledge Discovery and Data Mining.2020:354-367.
[27] CHEN Y X,LI D S,ZHANG P,et al.Cross-modal Ambiguity Learning for Multimodal Fake News Detection[C]//Procee-dings of the ACM Web Conference.2022:2897-2905.
[28] ZHOU Y,YANG Y,YING Q,et al.Multimodal Fake News Detection via CLIP-guided Learning[C]//Proceedings of the IEEE International Conference on Multimedia and Expo.2023:2825-2830.
[29] WANG J,ZHANG H,LIU C,et al.Fake News Detection via Multi-scale Semantic Alignment and Cross-modal Attention[C]//Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval.2024:2406-2410.
[30] KUMAR V.Generative Adversarial Networks-aided IntrusionDetection System[C]//Generative Adversarial Networks and Deep Learning.2024:79-98.
[31] HE P,ZHANG H,CAO S,et al.Cross-Modal Fake News Detection Method Based on Multi-Level Fusion Without Evidence[J].Algorithms,2025,18(7):426.
[32] LIU Y F,LIU Y K,LI Z L,et al.Modality Interactive Mixture-of-Experts for Fake News Detection[C]//Proceedings of the ACM on Web Conference 2025(WWW’25).2025:5139-5150.
[33] HU J,YANG M,TANG B B,et al.Integrating Message Content and Propagation Path for Enhanced False Information Detection Using Bidirectional Graph Convolutional Neural Networks[J].Applied Science,2025,15:3457.
[34] CHEN X Q,HUANG X Y,GAO Q,et al.Enhancing text-centric fake news detection via external knowledge distillation from LLMs[J].Neural networks:the official journal of the International Neural Network Society,2025,187:107377.
[35] RADFORD A,KIM J,HALLACY C,et al.Learning Transferable Visual Models from Natural Language Supervision[C]//Proceedings of the International Conference on Machine Lear-ning.2021:8748-8763.
[36] VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[C]//Proceedings of the 31st International Confe-rence on Neural Information Processing Systems.2017:6000-6010.
[37] VINOD N,GEOFFREY E H.Rectified Linear Units ImproveRestricted Boltzmann Machines Vinod Nair[C]//Proceedings of the 27th International Conference on Machine Learning.2010:807-814.
[38] MCCLELLAND J,RUMELHART D.Parallel Distributed Processing:Explorations in the Microstructure of Cognition:Psychological and Biological Models[M].MIT Press,1987.
[39] KULLBACK S,LEIBLER R A.On Information and Sufficiency[J].The Annals of Mathematical Statistics,1951,22(1):79-86.
[40] KINGMA D P,WELLING M.Auto-Encoding Variational Bayes[C]//Proceedings of the International Conference on Learning Representations.2014.
[41] ZUBIAGA A,LIAKATA M,PROCTER R.Exploiting Context for Rumour Detection in Social Media[M]//Lecture Notes in Computer Science.2017:109-123.
[42] SHU K,MAHUDESWARAN D,WANG S H,et al.FakeNewsNet:A data repository with news content,social context,and spatiotemporal information for studying fake news on social media[J].Big Data,2020,8(3):171-188.
[43] MADRY A,MAKELOV A,SCHMIDT L,et al.Towards Deep Learning Models Resistant to Adversarial Attacks[C]//Proceedings of the International Conference on Learning Representations.2017.
[44] YUAN C,MA Q,ZHOU W,et al.Jointly Embedding the Local and Global Relations of Heterogeneous Graph for Rumor Detection[C]//Proceedings of IEEE International Conference on Data Mining.2019:796-805.
[45] DIDRIK P K,JIMMY B.Adam:A Method for Stochastic Optimization[C]//Proceedings of the International Conference on Learning Representations Computer Science.2014.
[1] FENG Haoyu, ZHANG Yuxuan, LIU Zixuan, MENG Hua. Network Anomaly Traffic Detection Based on Deep Multi-instance Learning [J]. Computer Science, 2026, 53(8): 426-436.
[2] ZHU Bin, LI Xiaobin. AETC:Image Classification Model via Attention-based Topological Features Fusion [J]. Computer Science, 2026, 53(7): 24-33.
[3] ZHANG Yan, ZHOU Jian, HAN Lei, CHENG Chunling. Gate-controlled Agent Attention Mechanism-based Soft Prompt Transfer Method [J]. Computer Science, 2026, 53(7): 139-145.
[4] DING Zhijun, AI Fangju, LIU Aihuan. Document-level Event Argument Extraction Model Based on Hierarchical Dependency Aggregation and Event Enhancement [J]. Computer Science, 2026, 53(7): 156-167.
[5] LI Kaiju, YIN Chenyang, CHENG Zhangtao, LIU Xueting, ZHOU Fan. Causal Subgraph Learning for Cascade Popularity Prediction [J]. Computer Science, 2026, 53(7): 213-221.
[6] XIAO Yanxue, DENG Li, REN Zhengwei, WU Mengxin. iDSRformer:Node Load Prediction Model for High-performance Computing Cluster [J]. Computer Science, 2026, 53(7): 251-261.
[7] DUAN Haiying, WANG Baohui, HUANG He. Malicious Traffic Detection Method of ICMP Covert Channel Based on Baseline Features [J]. Computer Science, 2026, 53(6A): 250200069-11.
[8] LI Jie, WANG Baohui, ZHANG Jingyuan. DDoS Attack Detection Based on Attention Mechanism TCN-BiLSTM [J]. Computer Science, 2026, 53(6A): 250300060-9.
[9] ZHANG Shouyi, SHEN Qiang, GUO Yiran, WANG Hanyu. Rain and Fog Weather Object Detection Algorithm Based on Improved YOLOv8 Model [J]. Computer Science, 2026, 53(6A): 250300090-7.
[10] YANG Geer, WANG Xin, SUN Wei, WANG Xinge, HU Zhongrui, MENG Wenjun, ZHANG Junqiang, WU Xinghui, LIU Jinshan, YAN Yuming. Survey on Positional Encoding Algorithms in Deep Learning [J]. Computer Science, 2026, 53(6A): 250300107-16.
[11] WEI Wei, LI Bicheng, ZHU Zhenshui, ZUO Jun. Semantic Modeling and Co-attention Mechanism for Multimodal Sarcasm Detection Method [J]. Computer Science, 2026, 53(6A): 250400127-6.
[12] FENG Guang, LIN Jianzhong, ZHONG Ting, ZHOU Yuanhua, ZHENG Runting, LIU Tianxiang. Triple Extraction Based on Pixel Difference Convolutional Network and Attention Mechanism [J]. Computer Science, 2026, 53(6A): 250400136-10.
[13] CHEN Dianlong, LIU Tengbin, GAO Xiong, TIAN Zijian, ZHU Wenbing, ZOU Shun, WANG Qiang. Defect Detection of Transmission Line Fittings Based on Multiscale Feature Fusion Attention and Cross-layer Aggregation [J]. Computer Science, 2026, 53(6A): 250600110-7.
[14] SHEN Yingchun, FENG Xiaohan, LI Qian. Accurate Recognition of Dialect Based on CTC-Conformer Model [J]. Computer Science, 2026, 53(6A): 250600112-8.
[15] ZHONG Hao, KONG Qingxuan, CAI Xianqing, LI Zhizhong, SUN Hao. Intelligent Recognition Method Based on Multimodal Feature Fusion [J]. Computer Science, 2026, 53(6A): 250700065-10.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!