Computer Science ›› 2021, Vol. 48 ›› Issue (3): 27-39.doi: 10.11896/jsjkx.210100079

Special Issue: Advances on Multimedia Technology

• Advances on Multimedia Technology • Previous Articles     Next Articles

Adversarial Attacks and Defenses on Multimedia Models:A Survey

CHEN Kai, WEI Zhi-peng, CHEN Jing-jing, JIANG Yu-gang   

  1. School of Computer Science,Fudan University,Shanghai 201203,China
    Shanghai Key Laboratory of Intelligent Information,Shanghai 200433,China
  • Received:2021-01-10 Revised:2021-02-03 Online:2021-03-15 Published:2021-03-05
  • About author:CHEN Kai,born in 1998,postgraduate.His main research interests include ima-ge and video adversarial attack.
    JIANG Yu-gang,born in 1981,Ph.D,professor,Ph.D supervisor,is a member of China Computer Federation.His main research interests include multimedia content analysis,computer vision and robust & trustworthy AI.
  • Supported by:
    National Natural Science Foundation of China(62032006) and Science and Technology Commission of Shanghai Municipality(20511101000).

Abstract: In recent years,with the rapid development and wide application of deep learning,artificial intelligence is profoundly changing all aspects of social life.However,artificial intelligence models are also vulnerable to well-designed “adversarial examples”.By adding subtle perturbations that are imperceptible to humans on clean image or video samples,it is possible to generate adversarial examples that can deceive the model,which leads the multimedia model to make wrong decisions in the inference process,and bring serious security threat to the actual application and deployment of the multimedia model.In view of this,adversarial examples generation and defense methods for multimedia models have attracted widespread attention from both academic and industry.This paper first introduces the basic principles and relevant background knowledge of adversarial examples generation and defense.Then,it reviews the recent progress on both adversarial attack and defense on multimedia models.Finally,it summarizes the current challenges as well as the future directions for adversarial attacks and defenses.

Key words: Adversarial attack, Adversarial defense, Deep learning, Image adversarial sample, Video adversarial sample

CLC Number: 

  • TP18
[1]SUN Y,WANG X,TANG X.Deep learning face representation from predicting 10 000 classes[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2014:1891-1898.
[2]TAIGMAN Y,YANG M,RANZATO M A,et al.Deepface:Closing the gap to human-level performance in face verification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2014:1701-1708.
[3]REDMON J,FARHADI A.YOLO9000:better,faster,stronger[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:7263-7271.
[4]REN S,HE K,GIRSHICK R,et al.Faster r-cnn:Towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,39(6):1137-1149.
[5]SAON G,KUO H K J,RENNIE S,et al.The IBM 2015 English Conversational Telephone Speech Recognition System[C]//Sixteenth Annual Conference of the International Speech Communication Association.2015:3140-3144.
[6]SILVER D,SCHRITTWIESER J,SIMONYAN K,et al.Mastering the game of go without human knowledge[J].Nature,2017,550(7676):354-359.
[7]SZEGEDY C,ZAREMBA W,SUTSKEVER I,et al.Intriguing properties of neural networks[J].arXiv:1312.6199,2013.
[8]SU J,VARGAS D V,SAKURAI K.One pixel attack for fooling deep neural networks[J].IEEE Transactions on Evolutionary Computation,2019,23(5):828-841.
[9]WEI X,ZHU J,YUAN S,et al.Sparse adversarial perturbations for videos[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2019,33:8973-8980.
[10]MISHKIN D,MATAS J.All you need is a good init[J].arXiv:1511.06422,2015.
[11]MAAS A L,HANNUN A Y,NG A Y.Rectifier nonlinearities improve neural network acoustic models[C]//Proc.ICML.2013,30(1):3.
[12]CLEVERT D A,UNTERTHINER T,HOCHREITER S.Fast and accurate deep network learning by exponential linear units (elus)[J].arXiv:1511.07289,2015.
[13]SHARIF M,BHAGAVATULA S,BAUER L,et al.Accessorize to a crime:Real and stealthy attacks on state-of-the-art face recognition[C]//Proceedings of the 2016 ACM Sigsac Conference on Computer and Communications Security.2016:1528-1540.
[14]LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-basedlearning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[15]KRIZHEVSKY A,HINTON G.Learning multiple layers of features from tiny images[J].Handbook of Systemic Autoimmune Diseases,2009,1(4).
[16]RUSSAKOVSKY O,DENG J,SU H,et al.Imagenet large scale visual recognition challenge[J].International Journal of Computer Vision,2015,115(3):211-252.
[17]SOOMRO K,ZAMIR A R,SHAH M.UCF101:A dataset of101 human actions classes from videos in the wild[J].arXiv:1212.0402,2012.
[18]KUEHNE H,JHUANG H,GARROTE E,et al.HMDB:a large video database for human motion recognition[C]//2011 International Conference on Computer Vision.IEEE,2011:2556-2563.
[19]KAY W,CARREIRA J,SIMONYAN K,et al.The kinetics human action video dataset[J].arXiv:1705.06950,2017.
[20]GOODFELLOW I J,SHLENS J,SZEGEDY C.Explaining and harnessing adversarial examples[J].Stat,2015,1050:20.
[21]KURAKIN A,GOODFELLOW I,BENGIO S.Adversarial machine learning at scale[J].arXiv:1611.01236,2016.
[22]TRAMER F,KURAKIN A,PAPERNOT N,et al.Ensembleadversarial training:attacks and defenses[J].Stat,2018,1050:22.
[23]KURAKIN A,GOODFELLOW I J,BENGIO S.Adversarial examples in the physical world[J].arXiv:1607.02533,2016.
[24]MDRY A,MAKELOV A,SCHMIDT L,et al.Towards Deep Learning Models Resistant to Adversarial Attacks[J].Stat,2017,1050:9.
[25]PAPERNOT N,MCDANIEL P,JHA S,et al.The limitations of deep learning in adversarial settings[C]//2016 IEEE European Symposium on Security and Privacy (EuroS&P).IEEE,2016:372-387.
[26]MOOSAVI-DEZFOOLI S M,FAWZI A,FROSSARD P.Deep-fool:a simple and accurate method to fool deep neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:2574-2582.
[27]CARLINI N,WAGNER D.Towards evaluating the robustness of neural networks[C]//2017 IEEE Symposium on Security and Privacy (SP).IEEE,2017:39-57.
[28]CROCE F,HEIN M.Minimally distorted adversarial examples with a fast adaptive boundary attack[C]//International Confe-rence on Machine Learning.PMLR,2020:2196-2205.
[29]CROCE F,HEIN M.Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks[C]//International Conference on Machine Learning.PMLR,2020:2206-2216.
[30]ANDRIUSHCHENKO M,CROCE F,FLAMMARION N,et al.Square attack:a query-efficient black-box adversarial attack via random search[C]//European Conference on Computer Vision.Springer,Cham,2020:484-501.
[31]LIU Y,CHEN X,LIU C,et al.Delving into transferable adversarial examples and black-box attacks[J].arXiv:1611.02770,2016.
[32]DONG Y,LIAO F,PANG T,et al.Boosting adversarial attacks with momentum[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:9185-9193.
[33]POLYAKB T.Some methods of speeding up the convergence of iteration methods[J].USSR Computational Mathematics and Mathematical Physics,1964,4(5):1-17.
[34]LIN J,SONG C,HE K,et al.Nesterov accelerated gradient and scale invariance for adversarial attacks[J].arXiv:1908.06281,2019.
[35]KRIZHEVSKY A,SUTSKEVER I,HINTON G E.Imagenetclassification with deep convolutional neural networks[J].Communications of the ACM,2017,60(6):84-90.
[36]SIMONYAN K,ZISSERMAN A.Very deep convolutional net-works for large-scale image recognition[J].arXiv:1409.1556,2014.
[37]HE K,ZHANG X,REN S,et al.Identity mappings in deep residual networks[C]//European Conference on Computer Vision.Springer,Cham,2016:630-645.
[38]XIE C,ZHANG Z,ZHOU Y,et al.Improving transferability of adversarial examples with input diversity[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2019:2730-2739.
[39]LI Y,BAI S,ZHOU Y,et al.Learning transferable adversarial examples via ghost networks[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:11458-11465.
[40]CHEN P Y,ZHANG H,SHARMA Y,et al.Zoo:Zeroth order optimization based black-box attacks to deep neural networks without training substitute models[C]//Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security.2017:15-26.
[41]LAX P D,TERRELL M S.Calculus with applications[M].New York:Springer,2014.
[42]KINGMA D P,BA J.Adam:A method for stochastic optimization[J].arXiv:1412.6980,2014.
[43]ILYAS A,ENGSTROM L,ATHALYE A,et al.Black-box Adversarial Attacks with Limited Queries and Information[C]//Proceedings of the 35th International Conference on Machine Learning(ICML 2018).2018:2137-2146.
[44]WIERSTRA D,SCHAUL T,PETERS J,et al.Natural evolution strategies[C]//2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).IEEE,2008:3381-3387.
[45]SALIMANS T,HO J,CHEN X,et al.Evolution strategies as a scalable alternative to reinforcement learning[J].arXiv:1703.03864,2017.
[46]TU C C,TING P,CHEN P Y,et al.Autozoom:Autoencoder-based zeroth order optimization method for attacking black-box neural networks[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2019,33:742-749.
[47]NESTEROV Y,SPOKOINY V.Random gradient-free minimi-zation of convex functions[J].Foundations of Computational Mathematics,2017,17(2):527-566.
[48]BRENDEL W,RAUBER J,BETHGE M.Decision-based adversarial attacks:Reliable attacks against black-box machine learning models[J].arXiv:1712.04248,2017.
[49]CHENG M,LE T,CHENP Y,et al.Query-efficient hard-label black-box attack:An optimization-based approach[J].arXiv:1807.04457,2018.
[50]GROSSE K,MANOHARAN P,PAPERNOT N,et al.On the(statistical) detection of adversarial examples[J].arXiv:1702.06280,2017.
[51]FEINMAN R,CURTIN R R,SHINTRE S,et al.Detecting adve-rsarial samples from artifacts[J].arXiv:1703.00410,2017.
[52]METZEN J H,GENEWEIN T,FISCHERV,et al.On detecting adversarial perturbations[J].Stat,2017,1050:21.
[53]XU W,EVANS D,QI Y.Feature squeezing:Detecting adversa-rial examples in deep neural networks[J].arXiv:1704.01155,2017.
[54]MENG D,CHEN H.Magnet:a two-pronged defense against adversarial examples[C]//Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security.2017:135-147.
[55]LIAO F,LIANG M,DONG Y,et al.Defense against adversarial attacks using high-level representation guided denoiser[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:1778-1787.
[56]RONNEBERGER O,FISCHER P,BROX T.U-net:Convolu-tional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing andCompu-ter-Assisted Intervention.Springer,Cham,2015:234-241.
[57]ATHALYE A,CARLINI N.On the robustness of the cvpr 2018 white-box adversarial example defenses[J].arXiv:1804.03286,2018.
[58]XIE C,WU Y,MAATEN L,et al.Feature denoising for improving adversarial robustness[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2019:501-509.
[59]LI S,NEUPANE A,PAUL S,et al.Adversarial perturbationsagainst real-time video classification systems[J].arXiv:1807.00458,2018.
[60]GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Gene-rative adversarial nets[C]//Advances in Neural Information Processing Systems.2014:2672-2680.
[61]NAEH I,PONY R,MANNOR S.Flickering Adversarial At-tacks on Video Recognition Networks[J].arXiv:2002.05123,2020.
[62]CHEN Z,XIE L,PANG S,et al.Appending adversarial frames for universal video attack[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.2021:3199-3208.
[63]JIANG L,MA X,CHEN S,et al.Black-box adversarial attacks on video recognition models[C]//Proceedings of the 27th ACM International Conference on Multimedia.2019:864-872.
[64]WEI Z,CHEN J,WEI X,et al.Heuristic black-box adversarialattacks on video recognition models[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:12338-12345.
[65]YAN H,WEI X,LI B.Sparse black-box video attack with reinforcement learning[J].arXiv:2001.03754,2020.
[66]WILLIAMS R J.Simple statistical gradient-following algorithms for connectionist reinforcement learning[J].Machine Learning,1992,8(3/4):229-256.
[67]ZHANG H,ZHU L,ZHU Y,et al.Motion-Excited Sampler:Video Adversarial Attack with Sparked Prior[C]//European Conference on Computer Vision.Springer,Cham,2020:240-256.
[68]XIAO C,DENG R,LI B,et al.Advit:Adversarial frames identifier based on temporal consistency in videos[C]//Proceedings of the IEEE International Conference on Computer Vision.2019:3968-3977.
[69]JIA X,WEI X,CAO X.Identifying and resisting adversarial vi-deos using temporal consistency[J].arXiv:1909.04837,2019.
[70]LO S Y,PATELV M.Defending against multiple and unfore-seen adversarial videos[J].arXiv:2009.05244,2020.
[1] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[2] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[3] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[4] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[5] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[6] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[7] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[8] HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[9] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[10] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[11] ZHOU Hui, SHI Hao-chen, TU Yao-feng, HUANG Sheng-jun. Robust Deep Neural Network Learning Based on Active Sampling [J]. Computer Science, 2022, 49(7): 164-169.
[12] SU Dan-ning, CAO Gui-tao, WANG Yan-nan, WANG Hong, REN He. Survey of Deep Learning for Radar Emitter Identification Based on Small Sample [J]. Computer Science, 2022, 49(7): 226-235.
[13] WANG Jun-feng, LIU Fan, YANG Sai, LYU Tan-yue, CHEN Zhi-yu, XU Feng. Dam Crack Detection Based on Multi-source Transfer Learning [J]. Computer Science, 2022, 49(6A): 319-324.
[14] CHU Yu-chun, GONG Hang, Wang Xue-fang, LIU Pei-shun. Study on Knowledge Distillation of Target Detection Algorithm Based on YOLOv4 [J]. Computer Science, 2022, 49(6A): 337-344.
[15] ZHOU Zhi-hao, CHEN Lei, WU Xiang, QIU Dong-liang, LIANG Guang-sheng, ZENG Fan-qiao. SMOTE-SDSAE-SVM Based Vehicle CAN Bus Intrusion Detection Algorithm [J]. Computer Science, 2022, 49(6A): 562-570.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!