Computer Science ›› 2026, Vol. 53 ›› Issue (10): 77-91.doi: 10.11896/jsjkx.260500030
• Large Models and Multimodal Intelligent Perception • Previous Articles Next Articles
CHENG Cheng, ZHONG Han
CLC Number:
| [1] INSIGHTS S.Global Social Media Statistics Research Summary 2025 [EB/OL].(2025-01-15) [2026-05-20].https://www.smartinsights.com/social-media-marketing/social-media-strategy/new-global-social-media-research/. [2] GANGULY A,BIN EMRAN A N,PUSPO S S C,et al.Masonperplexity at Multimodal Hate Speech Event Detection 2024:Hate Speech and Target Detection Using Transformer Ensembles [C]//Proceedings of the 7th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text(CASE 2024).Stroudsburg:Association for Computational Linguistics,2024:125-131. [3] MADRIAZA P,HASSAN G,BROUILLETTE-ALARIE S,et al.Exposure to Hate in Online and Traditional Media:A Systematic Review and Meta-Analysis of the Impact of This Exposure on Individuals and Communities [J].Campbell Systematic Reviews,2025,21(1):e70018. [4] AMIN F,HORAIRA M A,SHAWON M T A,et al.Phan-tomTroupe@ CASE 2025:Multimodal Hate Speech Detection in Text-Embedded Memes using Instruction-Tuned LLMs [C]//Proceedings of the 8th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Texts.Stroudsburg:Association for Computational Linguistics,2025:133-138. [5] SHARMA S,ALAM F,AKHTAR M S,et al.Detecting andUnderstanding Harmful Memes:A Survey [J].arXiv:2205.04274,2022. [6] DANILEVSKY M,QIAN K,AHARONOV R,et al.A Survey of the State of Explainable AI for Natural Language Processing [C]//Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing.Stroudsburg:Association for Computational Linguistics,2020:447-459. [7] YIN S,FU C,ZHAO S,et al.A Survey on Multimodal LargeLanguage Models [J].National Science Review,2024,11(12):nwae403. [8] LEWIS P,PEREZ E,PIKTUS A,et al.Retrieval-AugmentedGeneration for Knowledge-Intensive NLP Tasks [J].Advances in Neural Information Processing Systems,2020,33:9459-9474. [9] ZHANG Z,ZHANG A,LI M,et al.Multimodal Chain-of-Thought Reasoning in Language Models [J].arXiv:2302.00923,2023. [10] PAN S,LUO L,WANG Y,et al.Unifying Large LanguageModels and Knowledge Graphs:A Roadmap [J].IEEE Transactions on Knowledge and Data Engineering,2024,36(7):3580-3599. [11] LI Y,DU Y,ZHOU K,et al.Evaluating Object Hallucination in Large Vision-Language Models [C]//Proceedings of the 2023 Conference on Empirical Methods in Natural Language Proces-sing.Stroudsburg:Association for Computational Linguistics,2023:292-305. [12] HE K,ZHANG X,REN S,et al.Deep Residual Learning forImage Recognition [C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE Press,2016:770-778. [13] DEVLIN J,CHANG M,LEE K,et al.BERT:Pre-training ofDeep Bidirectional Transformers for Language Understanding [C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Stroudsburg:Association for Computational Linguistics,2019:4171-4186. [14] PRAMANICK S,SHARMA S,DIMITROV D,et al.MOMENTA:A Multimodal Framework for Detecting Harmful Memes and Their Targets [C]//Findings of the Association for Computational Linguistics:EMNLP 2021.Stroudsburg:Association for Computational Linguistics,2021:4439-4455. [15] LEE R K,CAO R,FAN Z,et al.Disentangling Hate in Online Memes [C]//Proceedings of the 29th ACM International Conference on Multimedia.New York:ACM Press,2021:5138-5147. [16] PONNUSAMY R,RAJIAKODI S,SIVAGNANAM B,et al.Fuse-MD:A Culturally-Aware Multimodal Model for Detecting Misogyny Memes [J].Natural Language Processing Journal,2026,8:100197. [17] HUANG J,PAN J,WAN Z,et al.Evolver:Chain-of-Evolution Prompting to Boost Large Multimodal Models for Hateful Meme Detection [C]//Proceedings of the 31st International Confe-rence on Computational Linguistics.Stroudsburg:Association for Computational Linguistics,2025:7321-7330. [18] LIN H,LUO Z,GAO W,et al.Towards Explainable Harmful Meme Detection Through Multimodal Debate Between Large Language Models [C]//Proceedings of the ACM Web Confe-rence 2024.New York:ACM Press,2024:2359-2370. [19] ZHUANG Y,GUO K,WANG J,et al.I Know What YouMeme!Understanding and Detecting Harmful Memes with Multimodal Large Language Models [J].arXiv:2502.03456,2025. [20] MEI J,SUN M,CHEN J,et al.ExPO-HM:Learning to Explain-then-Detect for Hateful Meme Detection [J].arXiv:2510.08630,2025. [21] MEI J,CHEN J,YANG G,et al.Robust Adaptation of Large Multimodal Models for Retrieval Augmented Hateful Meme Detection [C]//Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.Stroudsburg:Association for Computational Linguistics,2025:23806-23828. [22] TZELEPI M,MEZARIS V.Improving Multimodal HatefulMeme Detection Exploiting LMM-Generated Knowledge [C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshop.New York:IEEE Press,2025:202-211. [23] KMAINASI M B,HASNAT A,HASAN M A,et al.MemeIntel:Explainable Detection of Propagandistic and Hateful Memes [C]//Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.Stroudsburg:Association for Computational Linguistics,2025:30263-30279. [24] LIU O S,NG P C,SOH D W,et al.Yes FLoReNce,I Will Do Better Next Time! Agentic Feedback Reasoning for Humorous Meme Detection [J].arXiv:2601.07232,2026. [25] CORMEN T H,LEISERSON C E,RIVEST R L,et al.Introduction to Algorithms [M].Cambridge:MIT Press,2022:530-560. [26] HU E J,SHEN Y,WALLIS P,et al.LoRA:Low-Rank Adaptation of Large Language Models [J].International Conference on Learning Representations(ICLR),2022,1(2):3-11. [27] PRAMANICK S,DIMITROV D,MUKHERJEE R,et al.Detecting Harmful Memes and Their Targets [C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing:Findings.Stroudsburg:Association for Computational Linguistics,2021:2788-2796. [28] SHAH S B,SHIWAKOTI S,CHAUDHARY M,et al.MemeCLIP:Leveraging CLIP Representations for Multimodal Meme Classification [C]//Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing.Stroudsburg:Association for Computational Linguistics,2024:17320-17332. [29] FERSINI E,GASPARINI F,RIZZI G,et al.SemEval-2022 Task 5:Multimedia Automatic Misogyny Identification [C]//Proceedings of the 16th International Workshop on Semantic Evaluation(SemEval-2022).Stroudsburg:Association for Computational Linguistics,2022:533-541. [30] ZHENG L,CHIANG W,SHENG Y,et al.Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena [J].Advances in Neural Information Processing Systems,2023,36:46595-46623. [31] LIU A,FENG B,WANG B,et al.DeepSeek-V2:A Strong,Economical,and Efficient Mixture-of-Experts Language Model [J].arXiv:2405.04434,2024. [32] MATHEW B,SAHA P,YIMAM S M,et al.HateXplain:ABenchmark Dataset for Explainable Hate Speech Detection [C]//Proceedings of the AAAI Conference on Artificial Intelligence.Palo Alto:AAAI Press,2021:14867-14875. [33] LIU Y,ITER D,XU Y,et al.G-Eval:NLG Evaluation using GPT-4 with Better Human Alignment [C]//Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing.Stroudsburg:Association for Computational Linguistics,2023:2510-2522. [34] VAN DER LEE C,GATT A,VAN MILTENBURG E,et al.Best Practices for the Human Evaluation of Automatically Ge-nerated Text [C]//Proceedings of the 12th International Confe-rence on Natural Language Generation.Stroudsburg:Association for Computational Linguistics,2019:355-368. [35] BAI S,CHEN K,LIU X,et al.Qwen2.5-VL Technical Report [J].arXiv:2502.10001,2025. [36] CHUNG H W,HOU L,LONGPRE S,et al.Scaling Instruction-Finetuned Language Models [J].Journal of Machine Learning Research,2024,25(70):1-53. [37] ZHENG Y,ZHANG R,ZHANG J,et al.LlamaFactory:Unified Efficient Fine-Tuning of 100+ Language Models [C]//Procee-dings of the 62nd Annual Meeting of the Association for Computational Linguistics:System Demonstrations.Stroudsburg:Association for Computational Linguistics,2024:400-410. [38] LOSHCHILOV I,HUTTER F.Decoupled Weight Decay Regularization [J].arXiv:1711.05101,2017. [39] ANDERSON P,HE X,BUEHLER C,et al.Bottom-Up andTop-Down Attention for Image Captioning and Visual Question Answering [C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE Press,2018:6077-6086. [40] KIELA D,FIROOZ H,MOHAN A,et al.The Hateful Memes Challenge:Detecting Hate Speech in Multimodal Memes [J].Advances in Neural Information Processing Systems,2020,33:2611-2624. [41] KIELA D,BHOOSHAN S,FIROOZ H,et al.Supervised Multimodal Bitransformers for Classifying Images and Text [J].ar-Xiv:1909.02950,2019. [42] LI L H,YATSKAR M,YIN D,et al.VisualBERT:A Simple and Performant Baseline for Vision and Language [J].arXiv:1908.03557,2019. [43] LU J,BATRA D,PARIKH D,et al.ViLBERT:PretrainingTask-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks [J].Advances in Neural Information Proces-sing Systems,2019,32:13-23. [44] CAO R,LEE R K,CHONG W,et al.Prompting for Multimodal Hateful Meme Classification [C]//Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing.Stroudsburg:Association for Computational Linguistics,2022:321-332. [45] CAO R,HEE M S,KUEK A,et al.Pro-Cap:Leveraging a Frozen Vision-Language Model for Hateful Meme Detection [C]//Proceedings of the 31st ACM International Conference on Multimedia.New York:ACM Press,2023:5244-5252. [46] ZHANG B,ZHANG P,DONG X,et al.Long-CLIP:Unlocking the Long-Text Capability of CLIP [C]//Proceedings of the European Conference on Computer Vision.Berlin:Springer,2024:310-325. [47] LIU H,LI C,LI Y,et al.Improved Baselines with Visual Instruction Tuning [C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.New York:IEEE Press,2024:26296-26306. [48] GRATTAFIORI A,DUBEY A,JAUHRI A,et al.The Llama 3Herd of Models [J].arXiv:2407.21783,2024. [49] MADAAN A,TANDON N,GUPTA P,et al.Self-refine:Iterative refinement with self-feedback [J].Advances in Neural Information Processing Systems,2023,36:46534-46594. [50] BESTA M,BLACH N,KUBICEK A,et al.Graph of thoughts:Solving elaborate problems with large language models [C]//Proceedings of the AAAI Conference on Artificial Intelligence.Palo Alto:AAAI Press,2024:17682-17690. |
| [1] | HU Yihui, LI Yan, GUO Tianxiang, LI Wentao, HU Xingchen. Ship Recognition Methods via Rule Construction from Distributed Incomplete Data [J]. Computer Science, 2026, 53(9): 136-144. |
| [2] | ZHAO Junlong, XU Bin, XIN Weiwei. Adversarial Knowledge Distillation Framework Integrating Temperature Removal Mechanism and Mutual Information Enhancement Strategy [J]. Computer Science, 2026, 53(9): 283-290. |
| [3] | ZHONG Rui, YAN Hongwei, LIU Jiawei. Lightweight Low-resolution Face Recognition via Hierarchical Dynamic Feature Generation Distillation [J]. Computer Science, 2026, 53(8): 209-218. |
| [4] | PING Fengqin, FU Xiaodong. Personalized Federated Learning for Concept and Label Distribution Drift [J]. Computer Science, 2026, 53(7): 433-441. |
| [5] | LI Siyu, QIAN Wenhua. HCKD:Lightweight Skin Lesion Classification Method Based on Dermoscopic Images [J]. Computer Science, 2026, 53(6A): 250600143-9. |
| [6] | FU Shiqi, ZHU Jinxia, XU Qichen, DU Zeyu. Dynamic Sparsity and Heterogeneous Knowledge Distillation for Top-k Recommendation [J]. Computer Science, 2026, 53(6A): 250700121-9. |
| [7] | SONG Jianhua, LIU Chun, ZHANG Yan. Lightweight Camouflaged Object Detection Model Based on Structured Knowledge Distillation [J]. Computer Science, 2026, 53(4): 299-307. |
| [8] | LI Yidan, CUI Jianying, XIONG Minghui. Category-Theoretic Semantic Representation: Systematic Review and Compositional Mechanism Analysis [J]. Computer Science, 2026, 53(4): 337-346. |
| [9] | PAN Jiahao, FENG Xiang, YU Huiqun. SM-PHT:Robust,Scalable,and Efficient Method for Multi-task Reinforcement Learning [J]. Computer Science, 2026, 53(4): 366-376. |
| [10] | WU Qiaorui, LUO Li, ZHAO Cairong. LLM-augmented Training Framework with Cycle-Consistency Constraints [J]. Computer Science, 2026, 53(4): 377-383. |
| [11] | LIU Zhenyu, YAO Dengfeng, ZHAO Yuan, MENG Ziwen, XU Cheng, LI Yan. Research Progress of Multimodal Machine Learning in Audio-Visual Semantic Analysis [J]. Computer Science, 2026, 53(10): 1-17. |
| [12] | WEN Zerui, JIANG Tian, HUANG Zijian, CUI Xiaohui. Section Sparse Attack:A More Powerful Sparse Attack Method [J]. Computer Science, 2026, 53(1): 323-330. |
| [13] | JIANG Yunliang, JIN Senyang, ZHANG Xiongtao, LIU Kaining, SHEN Qing. Multi-scale Multi-granularity Decoupled Distillation Fuzzy Classifier and Its Application inEpileptic EEG Signal Detection [J]. Computer Science, 2025, 52(9): 37-46. |
| [14] | DENG Jiayan, TIAN Shirui, LIU Xiangli, OUYANG Hongwei, JIAO Yunjia, DUAN Mingxing. Trajectory Prediction Method Based on Multi-stage Pedestrian Feature Mining [J]. Computer Science, 2025, 52(9): 241-248. |
| [15] | LIU Le, XIAO Rong, YANG Xiao. Application of Decoupled Knowledge Distillation Method in Document-level RelationExtraction [J]. Computer Science, 2025, 52(8): 277-287. |
|
||