Computer Science ›› 2026, Vol. 53 ›› Issue (10): 18-40.doi: 10.11896/jsjkx.260500079
• Large Models and Multimodal Intelligent Perception • Previous Articles Next Articles
ZHONG Yue1,2, GU Jieming3,4
CLC Number:
| [1] ACHIAM J,ADLER S,AGARWAL S,et al.GPT-4 technical report[J].arXiv:2303.08774,2023. [2] TEAM G,ANIL R,BORGEAUD S,et al.Gemini:A family of highly capable multimodal models[J].arXiv:2312.11805,2023. [3] LIU Z,LIANG H,LI B,et al.SynthVLM:Towards high-quality and efficient synthesis of image-caption datasets for vision-language models[C]//ACM International Conference on Multimedia.2025:12807-12814. [4] LI L,PENG J,CHEN H,et al.How to configure good in-context sequence for visual question answering[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.2024:26710-26720. [5] SHI Y,YANG S,ZHU X,et al.MedM-VL:What makes a goodmedical LVLM?[C]//International Workshop on Agentic AI for Medicine.2025:290-299. [6] LÜBBERSTEDT J,GUERRERO E R,UHLEMANN N,et al.V3LMA:Visual 3D-enhanced language model for autonomous driving[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.2025:4769-4778. [7] CHENG R,DING Y,CAO S,et al.PBI-Attack:Prior-guided bimodal interactive black-box jailbreak attack for toxicity maximization[C]//Conference on Empirical Methods in Natural Language Processing.2025:609-628. [8] ZHANG Z,SUN Z,ZHANG Z,et al.FC-Attack:Jailbreaking large vision-language models via auto-generated flowcharts[C]//Findings of the Association for Computational Linguistics:EMNLP.2025:9299-9316. [9] CAI X,LIU D,QU X,et al.Towards building model/prompt-transferable attackers against large vision-language models[J].Advances in Neural Information Processing Systems,2025,38:174022-174058. [10] HOSSAIN M Z,IMTEAJ A.SLADE:Shielding against dual exploits in large vision-language models[C]//IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition.2025:24244-24254. [11] LIU Q,WANG F,XIAO C,et al.VLM-Guard:Safeguarding vision-language models via fulfilling safety alignment gap[J].ar-Xiv:2502.10486,2025. [12] ZHANG Y,LI J,CAI L,et al.DAVSP:Safety alignment forlarge vision-language models via deep aligned visual safety prompt[C]//AAAI Conference on Artificial Intelligence.2026,40(44):38111-38119. [13] YAO Y,DUAN J,XU K,et al.A survey on large language mo-del(LLM) security and privacy:The good,the bad,and the ugly[J].High-Confidence Computing,2024,4(2):100211. [14] ZHANG R,LI H W,QIAN X Y,et al.On large language mo-dels safety,security,and privacy:A survey[J].Journal of Electronic Science and Technology,2025,23(1):100301. [15] DONG Y,MU R,ZHANG Y,et al.Safeguarding large language models:A survey[J].Artificial Intelligence Review,2025,58(12):382. [16] DAS B C,AMINI M H,WU Y.Security and privacy challenges of large language models:A survey[J].ACM Computing Surveys,2025,57(6):1-39. [17] DUARTE J D,CÂNDIDO G D,DE BRITTO FILHO J R A,et al.A systematic review of prompt injection attacks on large language models:Trends,taxonomy,evaluation,defenses and opportunities[J].IEEE Access,2026,14:12875-12899. [18] LIU X,ZHU Y,LAN Y,et al.Safety of multimodal large language models on images and texts[C]//International Joint Conference on Artificial Intelligence.2024:8151-8159. [19] WANG T,FANG Z,XUE H,et al.Large vision-language model security:A survey[C]//International Conference on Frontiers in Cyber Security.2024:3-22. [20] FAN Y,CAO Y,ZHAO Z,et al.Unbridled icarus:A survey of the potential perils of image inputs in multimodal large language model security[C]//International Conference on Systems,Man,and Cybernetics.2024:3428-3433. [21] RAHMAN M A,ALQAHTANI L,ALBOOQ A,et al.A survey on security and privacy of large multimodal deep learning mo-dels:Teaching and learning perspective[C]//Learning and Technology Conference.2024:13-18. [22] LI Z,WU X,DU H,et al.A survey of state of the art largevision language models:Benchmark evaluations and challenges[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.2025:1587-1606. [23] RADFORD A,KIM J W,HALLACY C,et al.Learning transferable visual models from natural language supervision[C]//International Conference on Machine Learning.2021:8748-8763. [24] LI J,LI D,XIONG C,et al.BLIP:Bootstrapping language-image pre-training for unified vision-language understanding and ge-neration[C]//International Conference on Machine Learning.2022:12888-12900. [25] CHAI Y,SUN H,FANG H,et al.MA-RLHF:Reinforcementlearning from human feedback with macro actions[C]//International Conference on Learning Representations.2025:26647-26680. [26] ZHU L,CHEN T,XU Q,et al.POPEN:Preference-based optimization and ensemble for lvlm-based reasoning segmentation[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.2025:30231-30240. [27] DENG J,DONG W,SOCHER R,et al.ImageNet:a large-scale hierarchical image database[C]//IEEE Conference on Computer Vision and Pattern Recognition.2009:248-255. [28] LIN T Y,MAIRE M,BELONGIE S,et al.Microsoft COCO:Common objects in context[C]//European Conference on Computer Vision.2014:740-755. [29] PLUMMER B A,WANG L,CERVANTES C M,et al.Flickr-30K entities:Collecting region-to-phrase correspondences for richer image-to-sentence models[C]//International Conference on Computer Vision.2015:2641-2649. [30] CHEN X,FANG H,LIN T Y,et al.Microsoft COCO captions:Data collection and evaluation server[J].arXiv:1504.00325,2015. [31] GOYAL Y,KHOT T,SUMMERS-STAY D,et al.Making the v in VQA matter:Elevating the role of image understanding in visual question answering[C]//IEEE Conference on Computer Vision and Pattern Recognition.2017:6904-6913. [32] MARINO K,RASTEGARI M,FARHADI A,et al.OK-VQA:A visual question answering benchmark requiring external knowledge[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:3195-3204. [33] GURARI D,LI Q,STANGL A J,et al.VizWiz grand challenge:Answering visual questions from blind people[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.2018:3608-3617. [34] LIU H,LI C,WU Q,et al.Visual instruction tuning[J].Advances in Neural Information Processing Systems,2023,36:34892-34916. [35] LU P,MISHRA S,XIA T,et al.Learn to explain:Multimodal reasoning via thought chains for science question answering[J].Advances in Neural Information Processing Systems,2022,35:2507-2521. [36] LIU Y,DUAN H,ZHANG Y,et al.MMBench:Is your multi-modal model an all-around player?[C]//European Conference on Computer Vision.2024:216-233. [37] ZHANG Y F,ZHANG H,TIAN H,et al.MME-RealWorld:Could your multimodal LLM challenge high-resolution real-world scenarios that are difficult for humans?[C]//International Conference on Learning Representations.2025:89655-89701. [38] XU P,SHAO W,ZHANG K,et al.LVLM-eHub:A comprehensive evaluation benchmark for large vision-language models[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2025,47(3):1877-1893. [39] SHAO W,LEI M,HU Y,et al.TinyLVLM-eHub:Towardscomprehensive and efficient evaluation for large vision-language models[J].IEEE Transactions on Big Data,2025,11(3):933-947. [40] GONG Y,RAN D,LIU J,et al.FigStep:Jailbreaking large vision-language models via typographic visual prompts[C]//AAAI Conference on Artificial Intelligence.2025:23951-23959. [41] LI M,LI L,YIN Y,et al.Red teaming visual language models[C]//Findings of the Association for Computational Linguistics:ACL.2024:3326-3342. [42] LIN H,LUO Z,WANG B,et al.GOAT-Bench:Safety insights to large multimodal models through meme-based social abuse[J].ACM Transactions on Intelligent Systems and Technology,2026,17(4):1-25. [43] LUO W,MA S,LIU X,et al.JailbreakV:A benchmark for assessing the robustness of multimodal large language models against jailbreak attacks[C]//Conference on Language Mode-ling.2024:1-21. [44] QI X,HUANG K,PANDA A,et al.Visual adversarial examples jailbreak aligned large language models[C]//AAAI Conference on Artificial Intelligence.2024:21527-21536. [45] LI Y,GUO H,ZHOU K,et al.Images are Achilles’ heel ofalignment:Exploiting visual vulnerabilities for jailbreaking multimodal large language models[C]//European Conference on Computer Vision.2024:174-189. [46] ZHANG H,SHAO W,LIU H,et al.B-AVIBench:Toward eva-luating the robustness of large vision-language model on black-box adversarial visual-instructions[J].IEEE Transactions on Information Forensics and Security,2025,20:1434-1446. [47] TU H,CUI C,WANG Z,et al.How many unicorns are in this image? A safety evaluation benchmark for vision LLMs[C]//European Conference on Computer Vision.2024:37-55. [48] LIU X,ZHU Y,GU J,et al.MM-SafetyBench:A benchmark for safety evaluation of multimodal large language models[C]//European Conference on Computer Vision.2024:386-403. [49] WANG X,YI X,JIANG H,et al.ToViLaG:Your visual-lan-guage generative model is also an evildoer[C]//Conference on Empirical Methods in Natural Language Processing.2023:3508-3533. [50] ZHANG Y,CHEN L,ZHENG G,et al.SPA-VL:A comprehensive safety preference alignment dataset for vision language models[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.2025:19867-19878. [51] CHEN Y,SIKKA K,COGSWELL M,et al.DRESS:Instructing large vision-language models to align and interact with humans via natural language feedback[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.2024:14239-14250. [52] MAZEIKA M,PHAN L,YIN X,et al.HarmBench:A standar-dized evaluation framework for automated red teaming and robust refusal[C]//International Conference on Machine Learning.2024:35181-35224. [53] LU D,PANG T,DU C,et al.Test-time backdoor attacks onmultimodal large language models[J].arXiv:2402.08577,2024. [54] LUO H,GU J,LIU F,et al.An image is worth 1000 lies:Transferability of adversarial images across prompts on vision-language models[C]//International Conference on Learning Representations.2023:23701-23722. [55] KONG J,FANG H,GUO S,et al.Neural ntidote:Class-wiseprompt tuning for purifying backdoors in pre-trained vision-language models[J].arXiv:2502.19269,2025. [56] PAPINENI K,ROUKOS S,WARD T,et al.BLEU:A method for automatic evaluation of machine translation[C]//Annual Meeting of the Association for Computational Linguistics.2002:311-318. [57] LIN C Y.ROUGE:a package for automatic evaluation of summaries[C]//Text Summarization Branches Out.2004:74-81. [58] BANERJEE S,LAVIE A.METEOR:An automatic metric forMT evaluation with improved correlation with human judgments[C]//ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization.2005:65-72. [59] VEDANTAM R,ZITNICK C L,PARIKH D.CIDEr:Consen-sus-based image description evaluation[C]//IEEE Conference on Computer Vision and Pattern Recognition.2015:4566-4575. [60] ANDERSON P,FERNANDO B,JOHNSON M,et al.Spice:Semantic propositional image caption evaluation[C]//European Conference on Computer Vision.2016:382-398. [61] REIMERS N,GUREVYCH I.Sentence-BERT:Sentence em-beddings using Siamese BERT-networks[C]//Conference on Empirical Methods in Natural Language Processing and International Joint Conference on Natural Language Processing.2019:3982-3992. [62] GEHMAN S,GURURANGAN S,SAP M,et al.RealToxicity-Prompts:Evaluating neural toxic degeneration in language models[C]//Findings of the Association for Computational Linguistics:EMNLP.2020:3356-3369. [63] GAO K,BAI Y,BAI J,et al.Adversarial robustness for visual grounding of multimodal large language models[C]//International Conference on Learning Representations Workshop on Reliable and Responsible Foundation Models.2024:1-8. [64] HEUSEL M,RAMSAUER H,UNTERTHINER T,et al.GANs trained by a two time-scale update rule converge to a local nash equilibrium[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.2017:6629-6640. [65] XU S,PANG L,ZHU Y,et al.Cross-modal safety mechanism transfer in large vision-language models[C]//International Conference on Learning Representations.2025:58315-58336. [66] KURAKIN A,GOODFELLOW I J,BENGIO S.Adversarial examples in the physical world[M]//Artificial intelligence safety and security.Chapman and Hall/CRC,2018:99-112. [67] CARLINI N,WAGNER D.Towards evaluating the robustness of neural networks[C]//IEEE Symposium on Security and Privacy.2017:39-57. [68] WANG Z,BOVIK A C,SHEIKH H R,et al.Image quality assessment:From error visibility to structural similarity[J].IEEE Transactions on Image Processing,2004,13(4):600-612. [69] ZHANG R,ISOLA P,EFROS A A,et al.The unreasonable effectiveness of deep features as a perceptual metric[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.2018:586-595. [70] ILYAS A,ENGSTROM L,ATHALYE A,et al.Black-box adversarial attacks with limited queries and information[C]//International Conference on Machine Learning.2018:2142-2151. [71] SZEGEDY C,ZAREMBA W,SUTSKEVER I,et al.Intriguing properties of neural networks [J].arXiv:1312.6199,2013. [72] WU T,LIN F,WANG G,et al.CoGA:A collaborative gray-box adversarial attack for multimodal language models[J].IEEE Transactions on Information Forensics and Security,2025,21:870-885. [73] ZHANG J,YI Q,SANG J.Towards adversarial attack on vision-language pre-training models[C]//ACM International Confe-rence on Multimedia.2022:5005-5013. [74] ZHANG Y,XIE R,CHEN J,et al.QAVA:Query-agnostic visual attack to large vision-language models[C]//Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics:Human Language Technologies.2025:10205-10218. [75] MADRY A,MAKELOV A,SCHMIDT L,et al.Towards deep learning models resistant to adversarial attacks[J].arXiv:1706.06083,2017. [76] SCHLARMANN C,HEIN M.On the adversarial robustness of multi-modal foundation models[C]//IEEE/CVF International Conference on Computer Vision.2023:3677-3685. [77] CROCE F,HEIN M.Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks[C]//International Conference on Machine Learning.2020:2206-2216. [78] ZHANG T,WANG L,ZHANG X,et al.Visual adversarial attack on vision-language models for autonomous driving[J].ar-Xiv:2411.18275,2024. [79] GAO K,BAI Y,GU J,et al.Inducing high energy-latency oflarge vision-language models with verbose images[C]//International Conference on Learning Representations.2024:17156-17182. [80] LIU D,CHEN B,HU W.Spatial-spectral homogeneous attacks on physical-world large vision-language models[C]//AAAI Conference on Artificial Intelligence.2026:7114-7122. [81] GUAN J,JIN H,WANG H.Crafting adversarial inputs for large vision-language models using black-box optimization[C]//Fin-dings of the Association for Computational Linguistics:EACL.2026:4698-4708. [82] YIN Z,YE M,ZHANG T,et al.VQAttack:Transferable adversarial attacks on visual question answering via pre-trained mo-dels[C]//AAAI Conference on Artificial Intelligence.2024:6755-6763. [83] CAI X,LIU D,GUAN R,et al.Imperceptible transfer attack on large vision-language models[C]//IEEE International Confe-rence on Acoustics,Speech and Signal Processing.2025:1-5. [84] HUANG H,ERFANI S,LI Y,et al.X-Transfer attacks:To-wards super transferable adversarial attacks on CLIP[C]//International Conference on Machine Learning.2025:1-31. [85] LIU D,LIU W,CAI X,et al.Generating transferable attacksacross large vision-language models using adversarial deformation learning[J].Pattern Recognition,2026,176:113194. [86] NIE S,ZHANG J,YAN J,et al.V-Attack:Targeting disentangled value features for controllable adversarial attacks on LVLMs[C]//IEEE/CVF International Conference on Computer Vision.2026. [87] ZHAO Y,PANG T,DU C,et al.On evaluating adversarial robustness of large vision-language models[J].Advances in Neural Information Processing Systems,2023,36:54111-54138. [88] WANG H,DONG K,ZHU Z,et al.Transferable multimodal attack on vision-language pre-training models[C]//IEEE Symposium on Security and Privacy.2024:1722-1740. [89] WU C H,SHAH R,KOH J Y,et al.Dissecting adversarial robustness of multimodal LM agents[C]//International Confe-rence on Learning Representations.2025:28362-28383. [90] LI Z,ZHAO X,WU D D,et al.A frustratingly simple yet highly effective attack baseline:Over 90% success rate against the strong black-box models of GPT-4.5/4o/o1[J] Advances in Neural Information Processing Systems,2025,38:45018-45051. [91] WANG L,ZHANG T,QU Y,et al.Black-box adversarial attack on vision language models for autonomous driving[J].arXiv:2501.13563,2025. [92] WANG X,JI Z,MA P,et al.InstructTA:Instruction-tuned targeted attack for large vision-language models[J].arXiv:2312.01886,2023. [93] WANG Y,ZHANG H,PAN H,et al.ADVEDM:Fine-grainedadversarial attack against VLM-based embodied agents[J].Advances in Neural Information Processing Systems,2025,38:136551-136575. [94] ZHANG X,LIU H,BAI L,et al.On the adversarial robustness of large vision-language models under visual token compression[J].arXiv:2601.21531,2026. [95] WANG Y,LIU C,QU Y,et al.Break the visual perception:Ad-versarial attacks targeting encoded visual tokens of large vision-language models[C]//ACM International Conference on Multimedia.2024:1072-1081. [96] MEI H,WANG Z,YOU S,et al.VEAttack:Downstream-agnostic vision encoder attack against large vision language models[C]//International Conference on Learning Representations.2026. [97] MEI H,WANG Z,XU C,et al.PA-Attack:Guiding gray-box attacks on LVLM vision encoders with prototypes and attention[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.2026. [98] CHAUDHARI S,AGGARWAL P,MURAHARI V,et al.RLHF deciphered:A critical analysis of reinforcement learning from human feedback for LLMs[J].ACM Computing Surveys,2025,58(2):1-37. [99] YU H,LIANG K,DUAN J,et al.TVChain:Leveraging textual-visual prompt chains for jailbreaking large vision-language mo-dels[C]//AAAI Conference on Artificial Intelligence.2026:27943-27951. [100] HUANG G,PENG Q,XU G,et al.LLaVAShield:Safeguarding multimodal multi-turn dialogues in vision-language models[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.2026. [101] WANG Y,ZHOU X,WANG Y,et al.Jailbreak large vision-language models through multi-modal linkage[C]//Annual Mee-ting of the Association for Computational Linguistics.2025:1466-1494. [102] ZENG Y,LIN H,ZHANG J,et al.How johnny can persuade LLMs to jailbreak them:Rethinking persuasion to challenge AI safety by humanizing LLMs[C]//Annual Meeting of the Association for Computational Linguistics.2024:14322-14350. [103] ROMBACH R,BLATTMANN A,LORENZ D,et al.High-re-solution image synthesis with latent diffusion models[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:10684-10695. [104] WANG R,LI J,WANG Y,et al.Ideator:Jailbreaking andbenchmarking large vision-language models using themselves[C]//IEEE/CVF International Conference on Computer Vision.2025:8875-8884. [105] YING Z,LIU A,ZHANG T,et al.Jailbreak vision language models via bi-modal adversarial prompt[J].IEEE Transactions on Information Forensics and Security,2025,20:7153-7165. [106] HAO H,WANG J,LIU A,et al.Activation manipulation at-tack:Penetrating and harmful jailbreak attack against large vision-language models[C]//AAAI Conference on Artificial Intelligence.2026:35481-35489. [107] SCHAEFFER R,VALENTINE D,BAILEY L,et al.Failures to find transferable image jailbreaks between vision-language mo-dels[C]//International Conference on Learning Representations.2025:44669-44704. [108] LIN R,PAREN A,YUAN S,et al.FORCE:Transferable visual jailbreaking attacks via feature over-reliance correction[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.2026. [109] CUI K,LI Y,WU Y,et al.Toward universal and transferable jailbreak attacks on vision-language models[C]//International Conference on Learning Representations.2026. [110] GULYAMOV S,GULYAMOV S,RODIONOV A,et al.Prompt injection attacks in large language models and AI agent systems:A comprehensive review of vulnerabilities,attack vectors,and defense mechanisms[J].Information,2026,17(1):54. [111] KIMURA S,TANAKA R,MIYAWAKI S,et al.Empirical ana-lysis of large vision-language models against goal hijacking via visual prompt injection[J].arXiv:2408.03554,2024. [112] KWON H,PAK W.Text-based prompt injection attack usingmathematical functions in modern large language models[J].Electronics,2024,13(24):5008. [113] CLUSMANN J,FERBER D,WIEST I C,et al.Prompt injection attacks on vision language models in oncology[J].Nature Communications,2025,16(1):1239. [114] LING C,HU K,LIU H,et al.Physical prompt injection attacks on large vision-language models[J].arXiv:2601.17383,2026. [115] WANG L,YING Z,ZHANG T,et al.Manipulating multimodal agents via cross-modal prompt injection[C]//ACM Internatio-nal Conference on Multimedia.2025:10955-10964. [116] BAGDASARYAN E,HSIEH T Y,NASSI B,et al.Abusingimages and sounds for indirect instruction injection in multi-modal LLMs[J].arXiv:2307.10490,2023. [117] WANG Z,SIU V,YE Z,et al.AgentVigil:Automatic black-box red-teaming for indirect prompt injection against LLM agents[C]//Findings of the Association for Computational Linguistics:EMNLP.2025:23159-23172. [118] LI Y,CAO Y,WANG D,et al.AgentTypo:Adaptive typogra-phic prompt injection attacks against black-box multimodal agents[J].arXiv:2510.04257,2025. [119] LIU Y,JIA Y,GENG R,et al.Formalizing and benchmarking prompt injection attacks and defenses[C]//USENIX Security Symposium.2024:1831-1847. [120] YI J,XIE Y,ZHU B,et al.Benchmarking and defending against indirect prompt injection attacks on large language models[C]//ACM SIGKDD Conference on Knowledge Discovery and Data Mining.2025:1809-1820. [121] CUI Y,PAN S,LIU Y,et al.VortexPIA:Indirect prompt injection attack against LLMs for efficient extraction of user privacy[C]//Findings of the Association for Computational Linguistics:EACL.2026:587-609. [122] ZHAN Q,LIANG Z,YING Z,et al.InjecAgent:Benchmarking indirect prompt injections in tool-integrated large language mo-del agents[C]//Findings of the Association for Computational Linguistics:ACL.2024:10471-10506. [123] DEBENEDETTI E,ZHANG J,BALUNOVIC M,et al.AgentDojo:A dynamic environment to evaluate prompt injection attacks and defenses for LLM agents[J].Advances in Neural Information Processing Systems,2024,37:82895-82920. [124] WANG X,BLOCH J,SHAO Z,et al.WebInject:Prompt injection attack to web agents[C]//Conference on Empirical Me-thods in Natural Language Processing.2025:2010-2030. [125] SAGER P J,MEYER B,YAN P,et al.A comprehensive survey of agents for computer use:Foundations,challenges,and future directions[J/OL].Journal of Artificial Intelligence Research,2026,85.https://dl.acm.org/doi/epdf/10.1613/jair.1.19490. [126] CAO T,LIM B,LIU Y,et al.VPI-Bench:Visual prompt injection attacks for computer-use agents[C]//International Confe-rence on Learning Representations.2026. [127] GU T,DOLAN-GAVITT B,GARG S.BadNets:Identifying vulnerabilities in the machine learning model supply chain[J].ar-Xiv:1708.06733,2017. [128] PAN X,ZHANG M,SHENG B,et al.Hidden trigger backdoor attack on NLP models via linguistic style manipulation[C]//USENIX Security Symposium.2022:3611-3628. [129] YUAN Z,SHI J,ZHOU P,et al.BadToken:Token-level backdoor attacks to multi-modal large language models[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.2025:29927-29936. [130] YE Z,ZHANG Y,SHI W,et al.VisualTrap:A stealthy backdoor attack on GUI agents via visual grounding manipulation[C]//Conference on Language Modeling.2025:1-17. [131] YIN Z,YE M,CAO Y,et al.Shadow-activated backdoor attacks on multimodal large language models[C]//Findings of the Association for Computational Linguistics:ACL.2025:4808-4829. [132] LI J,XU B,CHEN S,et al.IAG:Input-aware backdoor attack on VLM-based visual grounding[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.2026. [133] DU H,LIU S,ZHENG L,et al.Privacy in fine-tuning large language models:Attacks,defenses,and future directions[C]//Pacific-Asia Conference on Knowledge Discovery and Data Mi-ning.2025:326-344. [134] LYU W,PANG L,MA T,et al.TrojVLM:Backdoor attackagainst vision language models[C]//European Conference on Computer Vision.2024:467-483. [135] LIANG S,LIANG J,PANG T,et al.Revisiting backdoor attacks against large vision-language models from domain shift[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.2025:9477-9486. [136] BAI J,GAO K,MIN S,et al.BadCLIP:Trigger-aware prompt learning for backdoor attacks on CLIP[C]//IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition.2024:24239-24250. [137] LIANG J,LIANG S,LIU A,et al.VL-Trojan:Multimodal in-struction backdoor attacks against autoregressive visual language models[J].International Journal of Computer Vision,2025,133(7):3994-4013. [138] WANG Z,PANG G,MIAO W,et al.MTAttack:Multi-target backdoor attacks against large vision-language models[C]//AAAI Conference on Artificial Intelligence.2026:10440-10448. [139] ZHAN Q,HA H,YANG R,et al.BEAT:Visual backdoor attacks on VLM-based embodied agents via contrastive trigger learning[C]//International Conference on Learning Representations.2026. [140] LI Y,LI T,CHEN K,et al.BadEdit:Backdooring large language models by model editing[C]//International Conference on Learning Representations.2024:26117-26134. [141] GUO D,HU M,GUAN Z,et al.Backdoor in seconds:Unlocking vulnerabilities in large pre-trained models via model editing[C]//ACM International Conference on Information and Knowledge Management.2025:750-760. [142] TONG Z,SONG Y,WANG J,et al.VideoMAE:Masked au-toencoders are data-efficient learners for self-supervised video pre-training[J].Advances in Neural Information Processing Systems,2022,35:10078-10093. [143] LIU Z,ZHANG H.Stealthy backdoor attack in self-supervised learning vision encoders for large vision language models[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.2025:25060-25070. [144] ZHANG J,YE J,MA X,et al.AnyAttack:Towards large-scale self-supervised adversarial attacks on vision-language models[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.2025:19900-19909. [145] LEE S,KIM J,PAK W.Mind mapping prompt injection:Visual prompt injection attacks in modern large language models[J].Electronics,2025,14(10):1907. [146] LYU W,YAO J,GUPTA S,et al.Backdooring vision-language models with out-of-distribution data[C]//International Confe-rence on Learning Representations.2025:87511-87529. [147] WANG B,QIAN S,XU C.Invisible backdoor attack with siamese tuning on pre-trained vision-language models[J].IEEE Transactions on Multimedia,2026,28:1663-1676. [148] SCHLARMANN C,SINGH N D,CROCE F,et al.RobustCLIP:Unsupervised adversarial fine-tuning of vision embeddings for robust large vision-language models[C]//International Conference on Machine Learning.2024:43685-43704. [149] YU L,ZHANG H,XU C.Text-guided attention is all you need for zero-shot robustness in vision-language models[J].Advances in Neural Information Processing Systems,2024,37:96424-96448. [150] DONG J,KONIUSZ P,QU X,et al.Stabilizing modality gap & lowering gradient norms improve zero-shot adversarial robustness of VLMs[C]//ACM SIGKDD Conference on Knowledge Discovery and Data Mining.2025:236-247. [151] HOSSAIN M Z,IMTEAJ A.Sim-CLIP:Unsupervised siameseadversarial fine-tuning for robust and semantically-rich vision-language models[C]//International Joint Conference on Neural Networks.2026. [152] MAO C,GENG S,YANG J,et al.Understanding zero-shot adversarial robustness for large-scale models[C]//International Conference on Learning Representations.2023:1-19. [153] WANG S,ZHANG J,YUAN Z,et al.Pre-trained model guided fine-tuning for zero-shot adversarial robustness[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.2024:24502-24511. [154] CUI Y,GUAN X,XIONG Z,et al.AGFT:Alignment-guidedfine-tuning for zero-shot adversarial robustness of vision-language models[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.2026. [155] LU L,GU X,PANG S,et al.E2AT:Multimodal jailbreak defense via dynamic joint optimization for multimodal large language models[J].arXiv:2503.04833,2025. [156] JI J,CHEN X,PAN R,et al.Safe RLHF-V:Safe reinforcement learning from multi-modal human feedback[J].Advances in Neural Information Processing Systems,2025,38:46146-46182. [157] CAI W,LIU S,ZHAO J,et al.When safe unimodal inputs collide:Optimizing reasoning chains for cross-modal safety in multimodal large language models[C]//AAAI Conference on Artificial Intelligence.2026:35322-35330. [158] NGUYEN H V,PARK H,YOO N,et al.Enhancing largevision-language models for multimodal defect detection via SFT-GRPO reinforcement learning[C]//International Conference on Artificial Intelligence in Information and Communication.2026:1517-1522. [159] ZONG Y,BOHDAL O,YU T,et al.Safety fine-tuning at(almost) no cost:A baseline for vision large language models[C]//International Conference on Machine Learning.2024:62867-62891. [160] SCHULMAN J,WOLSKI F,DHARIWAL P,et al.Proximalpolicy optimization algorithms[J].arXiv:1707.06347,2017. [161] RAFAILOV R,SHARMA A,MITCHELL E,et al.Direct pre-ference optimization:Your language model is secretly a reward model[J].Advances in Neural Information Processing Systems,2023,36:53728-53741. [162] CUI J,CHIANG W L,STOICA I,et al.OR-Bench:An over-refusal benchmark for large language models[C]//International Conference on Machine Learning.2025:11515-11542 [163] YU J,LU X,WANG C,et al.LVLM-MIR:Large vision-lan-guage model with parameter-efficient fine-tuning for multimodal interleaved reasoning[C]//ACM International Conference on Multimedia.2025:13806-13812. [164] CHE C,WANG Z,YANG P,et al.LoRA in LoRA:Towards parameter-efficient architecture expansion for continual visual instruction tuning[C]//AAAI Conference on Artificial Intelligence.2026:19978-19986. [165] ZHAO S,WU X,NGUYEN C D T,et al.Unlearning backdoor attacks for llms with weak-to-strong knowledge distillation[C]//Findings of the Association for Computational Linguistics:ACL.2025:4937-4952. [166] KIM J,SONG M,NA S H,et al.Obliviate:Neutralizing task-ag-nostic backdoors within the parameter-efficient fine-tuning paradigm[C]//Findings of the Association for Computational Linguistics:NAACL.2025:1288-1307. [167] XUN Y,LIANG S,JIA X,et al.Robust anti-backdoor instruction tuning in LVLMs[J].arXiv:2506.05401,2025. [168] SUN Z,CONG T,LIU Y,et al.PEFTGuard:Detecting backdoor attacks against parameter-efficient fine-tuning[C]//IEEE Symposium on Security and Privacy.2025:1713-1731. [169] WANG Y,LIU X,LI Y,et al.AdaShield:Safeguarding multimodal large language models from structure-based attack via adaptive shield prompting[C]//European Conference on Computer Vision.2024:77-94. [170] ZHAO Y,ZHENG X,LUO L,et al.BlueSuffix:Reinforced blue teaming for vision-language models against jailbreak attacks[C]//International Conference on Learning Representations.2025:35443-35462. [171] ZHAO W,LI Z,LI Y,et al.Zero-shot defense against toxic images via inherent multimodal alignment in LVLMs[C]//Findings of the Association for Computational Linguistics:EMNLP.2025:14232-14246. [172] JIANG Y,GAO X,PENG T,et al.HiddenDetect:Detecting jailbreak attacks against large vision-language models via monitoring hidden states[C]//Annual Meeting of the Association for Computational Linguistics.2025:14880-14893. [173] NIE W,GUO B,HUANG Y,et al.Diffusion models for adversarial purification[C]//International Conference on Machine Learning.2022:16805-16827. [174] ZHANG Y,XIE R,HUANG Y,et al.Fighting fire with fire(F3):A training-free and efficient visual adversarial example purification method in LVLMs[C]//ACM International Confe-rence on Multimedia.2025:12266-12275. [175] JIANG W,LIANG K,RONG X,et al.PurMM:Attention-guided test-ti〗me backdoor purification in multimodal large Language models[C]//AAAI Conference on Artificial Intelligence.2026:35562-35570. [176] ZHANG X,ZHANG C,LI T,et al.JailGuard:A universal detection framework for prompt-based attacks on LLM systems[J].ACM Transactions on Software Engineering and Methodology,2025,35(1):1-40. [177] ZHANG J,MA X,WANG X,et al.Adversarial prompt tuning for vision-language models[C]//European Conference on Computer Vision.2024:56-72. [178] LI L,GUAN H,QIU J,et al.One prompt word is enough to boost adversarial robustness for pre-trained vision-language models[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.2024:24408-24419. [179] ZHOU C,YUE H,YAN M,et al.PromptGuard:Safeguarding large vision-language models via adversarial prompt tuning[J].Knowledge-Based Systems,2026,338:115498. [180] GAO J,PI R,HAN T,et al.CoCA:Regaining safety-awareness of multimodal large language models with constitutional calibration[C]//Conference on Language Modeling.2024:1-21. [181] WANG P,ZHANG D,LI L,et al.InferAligner:Inference-time alignment for harmlessness through cross-model guidance[C]//Conference on Empirical Methods in Natural Language Proces-sing.2024:10460-10479. [182] WANG W,MA Z,ZHENG H,et al.Self-aware safety augmentation:Leveraging internal semantic understanding to enhance safety in vision-language models[C]//ACM International Conference on Multimedia.2025:11199-11208. [183] ZHOU Q,LI T,GUO Q,et al.Defending LVLMs against vision attacks through partial-perception supervision[C]//Internatio-nal Conference on Machine Learning.2025:79254-79280. [184] CUNNINGHAM H,WEI J,WANG Z,et al.Constitutional classifiers++:Efficient production-grade defenses against universal jailbreaks[C]//International Conference on Learning Representations.2026. [185] GOU Y,CHEN K,LIU Z,et al.Eyes closed,safety on:Protecting multimodal LLMs via image-to-text transformation[C]//European Conference on Computer Vision.2024:388-404. [186] LIU Q,SHANG C,LIU L,et al.Unraveling and mitigating safety alignment degradation of vision-language models[C]//Findings of the Association for Computational Linguistics:ACL.2025:3631-3643. [187] PAREKH J,KHAYATAN P,SHUKOR M,et al.Learning to steer:Input-dependent steering for multimodalLLMs[J].Advances in Neural Information Processing Systems,2026,38:159799-159834. [188] WU L,WANG M,XU Z,et al.Automating steering for safe multimodal large language models[C]//Conference on Empirical Methods in Natural Language Processing.2025:792-814. [189] ZHU X,ZHU B,WANG S,et al.Principled steering via null-space projection for jailbreak defense in vision-language models[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.2026:22889-22899. [190] CHEN B,WANG C,YANG L,et al.LVLM-FDA:Protectinglarge vision-language models via fast detection of malicious attempts[C]//International Conference on Knowledge Science,Engineering and Management.2025:17-34. [191] ZHU K,YANG X,WANG J,et al.MELON:Provable defense against indirect prompt injection attacks in AI agents[C]//International Conference on Machine Learning.2025:80310-80329. [192] PHUTE M,HELBLING A,HULL M,et al.LLM self defense:By self examination,LLMs know they are being tricked[J].ar-Xiv:2308.07308,2023. [193] MARKOV T,ZHANG C,AGARWAL S,et al.A holistic Approach to undesired content detection in the real world[C]//AAAI Conference on Artificial Intelligence.2023:15009-15018. [194] HELFF L,FRIEDRICH F,BRACK M,et al.LlavaGuard:VLM-based safeguard for vision dataset curation and safety assessment[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.2024:8322-8326. [195] WANG Z,YANG F,WANG L,et al.Self-Guard:Empower the LLM to safeguard itself[C]//Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2024:1648-1668. [196] PI R,HAN T,ZHANG J,et al.MLLM-Protector:EnsuringMLLM’s safety without hurting performance[C]//Conference on Empirical Methods in Natural Language Processing.2024:16012-16027. [197] LI B,ZHANG Y,GUO D,et al.LLaVA-OneVision:Easy visual task transfer[J].arXiv:2408.03326,2024. [198] WANG Z,NAGARAJA N,ZHANG L,et al.To protect theLLM agent against the prompt injection attack with polymorphic prompt[C]//IEEE/IFIP International Conference on Dependable Systems and Networks-Supplemental Volume.2025:22-28. [199] RUAN J,YUAN W,GAO X,et al.VLRMBench:A comprehensive and challenging benchmark for vision-language reward mo-dels[C]//IEEE/CVF International Conference on Computer Vision.2025:3163-3173. [200] HARTVIGSEN T,GABRIEL S,PALANGI H,et al.ToxiGen:A large-scale machine-generated dataset for adversarial and implicit hate speech detection[C]//Annual Meeting of the Association for Computational Linguistics.2022:3309-3326. [201] WANG Z,ZHAO J,ZHANG L,et al.Targeted and transferable adversarial attacks on large vision-language models via two-stage region-specific perturbations[C]//International Confe-rence on Trust,Security and Privacy in Computing and Communications.2025:282-289. [202] ZHANG C,ZHOU L,XU X,et al.Adversarial attacks of vision tasks in the past 10 years:A survey[J].ACM Computing Surveys,2025,58(2):1-42. [203] XIE Y,YI J,SHAO J,et al.Defending ChatGPT against jailbreak attack via self-reminders[J].Nature Machine Intelligence,2023,5(12):1486-1496. [204] SHAPIRA N,WENDLER C,YEN A,et al.Agents of chaos[J].arXiv:2602.20021,2026. [205] TANG P,XIN H,ZHANG X,et al.SafetyReminder:Reviving delayed safety awareness of vision-language models to defend against jailbreak attacks[C]//AAAI Conference on Artificial Intelligence.2026:33223-33231. |
| [1] | CHEN Quantao, ZHANG Yangsen, WANG Pu, GUO Yalong. Survey on Security Risks and Mitigation Strategies for Generative Artificial Intelligence [J]. Computer Science, 2026, 53(7): 381-396. |
| [2] | WANG Jia, GAN Yongqiang. Multivariate Mimic Voting Method Based on Anomaly Perception [J]. Computer Science, 2026, 53(7): 414-421. |
| [3] | ZHENG Haibin, LIN Xiuhao, HAN Ye, CHEN Jinyin, LI Beibei. Black-box Physical Adversarial Attack Against Multimodal Object Detector [J]. Computer Science, 2026, 53(6A): 250700023-10. |
| [4] | CHEN Jun, TAO Wei, BAO Lei, TAO Qing. Momentum Method with Monotonical Coordinate-wise Step-sizes for Adversarial Attacks [J]. Computer Science, 2026, 53(5): 426-434. |
| [5] | CUI Tao, SHEN Junxia, CHEN Lin, ZHANG Yuntao, CHEN Monan. Technologies for Evaluating Defense Effectiveness of Endogenous Security Information Systems Based onAttack Graphs [J]. Computer Science, 2026, 53(5): 435-445. |
| [6] | DING Yan, DING Hongfa, YU Muran, JIANG Heling. Survey of Backdoor Attacks and Defenses on Graph Neural Network [J]. Computer Science, 2026, 53(3): 1-22. |
| [7] | ZHANG Wanyou, SONG Lipeng. Cloud Email Defense Resource Allocation Method Based on User Behavior [J]. Computer Science, 2026, 53(2): 442-453. |
| [8] | YAN Leiming, JI Lijing, WNAG Jinwei. Stealthy Backdoor Attack with Sensitivity-based Optimization Under Low Poisoning Rate [J]. Computer Science, 2026, 53(10): 455-463. |
| [9] | YI Xiaoyin, CHEN Long, ZHU Xiaofei. Stable Structural Representation and Distribution-level Relative Optimization for Cross-domain Transferable Adversarial Attacks [J]. Computer Science, 2026, 53(10): 476-485. |
| [10] | ZHOU Qiang, LI Zhe, TAO Wei, TAO Qing. Adaptive Box-constraint Optimization Method for Adversarial Attacks [J]. Computer Science, 2026, 53(1): 404-412. |
| [11] | HUANG Rong, TANG Yingchun, ZHOU Shubo , JIANG Xueqin. Composite Trigger Backdoor Attack Combining Visual and Textual Features [J]. Computer Science, 2026, 53(1): 382-394. |
| [12] | CHEN Jun, ZHOU Qiang, BAO Lei, TAO Qing. Linear Interpolation Method for Adversarial Attack [J]. Computer Science, 2025, 52(8): 403-410. |
| [13] | ZHU Keda, CAI Ruijie, LIU Shengli. Large Scale Network Defense Algorithm Based on Temporal Network Flow Watermarking Technology [J]. Computer Science, 2025, 52(6A): 240900110-6. |
| [14] | KANG Kai, WANG Jiabao, XU Kun. Balancing Transferability and Imperceptibility for Adversarial Attacks [J]. Computer Science, 2025, 52(6): 381-389. |
| [15] | CHEN Yanfeng, FENG Zhiwei, DENG Qingxu, WANG Yan. Survey of Sensor Attack Defense Strategies for Cyber Physical Systems [J]. Computer Science, 2025, 52(4): 4-13. |
|
||