Computer Science

Computer Science ›› 2023, Vol. 50 ›› Issue (9): 52-61.doi: 10.11896/jsjkx.230500235

• Data Security • Previous Articles     Next Articles

Research Progress of Backdoor Attacks in Deep Neural Networks

HUANG Shuxin, ZHANG Quanxin, WANG Yajie, ZHANG Yaoyuan, LI Yuanzhang   

  1. School of Computer Science & Technology,Beijing Institute of Technology,Beijing 100081,China
  • Received:2023-05-31 Revised:2023-06-24 Online:2023-09-15 Published:2023-09-01
  • About author:HUANG Shuxin,born in 1998,postgra-duate.Her main research interests include backdoor attacks and defences,and so on.
    LI Yuanzhang,born in 1978,Ph.D,associate professor.His main research intere-sts include mobile computing and information security.
  • Supported by:
    National Key Research and Development Program of China(2022YFB2701500) and National Natural Science Foundation of China(NSFC61876019).

PDF (PC)

2102

Abstract: In recent years,deep neural networks(DNNs) have developed rapidly,and their applications involve many fields,including auto autonomous driving,natural language processing,facial recognition and so on,which have brought a lot of convenience to people's life.However,the growth of DNNs has brought some security concerns.In recent years,DNNs have been shown to be vulnerable to backdoor attacks,mainly due to their low transparency and poor interpretability,allowing attackers to to swoop in.In this paper,the potential security and privacy risks in neural network applications are revealed by reviewing the research work related to neural network backdoor attacks,and the importance of research in the field of backdoor is emphasized.This paper first briefly introduces the threat model of neural network backdoor,then the neural network backdoor attack is divided into two categories:the backdoor attack based on poisoning and the backdoor attack without poisoning,and the poisoning attack can be subdivided into multiple categories.It aggregates available resources about backdoor attack,and analyzes the development of backdoor on neural network and the future development trend of backdoor attack is prospected.

Key words: Backdoor attack, Neural network, Machine learning, Poison attack, Non-poison attack

CLC Number: 

  • TP309.2
[1]YANG L C,BOTTOU L,BENGIO Y,et al.Gradient-basedlearning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[2]KRIZHEVSKY A,SUTSKEVER I,HINTON G E.Imagenetclassification with deep convolutional neural networks[J].Communications of the ACM,2017,60(6):84-90.
[3]SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].arXiv:1409.1556,2014.
[4]SZEGEDY C,LIU W,JIA Y,et al.Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:1-9.
[5]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[6]WANG K J,ZHAO Y D,XING X L.Research progress of deep learning in the field of autonomous vehicles [J].Journal of Intelligent Systems,2018,13(1):55-69.
[7]TIAN Y,PEI K,JANA S,et al.Deeptest:Automated testing of deep-neural-network-driven autonomous cars[C]//Proceedings of the 40th International Conference on Software Engineering.2018:303-314.
[8]LI J,MENG S G,FAN Q C,et al.Design and implementation of Access Control System based on Face Recognition [J].Automation and Information Engineering,2013,34(6):30-34.
[9]WANG M,DENG W.Deep face recognition:A survey[J].Neurocomputing,2021,429:215-244.
[10]HUANG L W,JIANG B T,LV S Y,et al.Review of recommen-dation systems based on Deep learning [J].Chinese Journal of Computers,2018,41(7):1619-1647.
[11]ZOU J,HAN Y,SO S S.Overview of artificial neural networks[J].Artificial Neural Networks:Methods and Applications,2009,148:14-22.
[12]GU T,DOLAN-GAVITT B,GARG S.Badnets:Identifyingvulnerabilities in the machine learning model supply chain[J].arXiv:1708.06733,2017.
[13]MIJWEL M M.Artificial neural networks advantages and disadvantages[J/OL].https//www.linkedin.com/pulse/artificial-neuralnetWork.
[14]SALEM X C A,ZHANG M.Badnl:Backdoor attacks against nlp models[C]//ICML 2021 Workshop on Adversarial Machine Learning.2021.
[15]SUN L.Natural backdoor attack on text data[J].arXiv:2006.16176,2020.
[16]LIU Y,MA S,AAFER Y,et al.Trojaning attack on neural networks[C]//25th Annual Network And Distributed System Security Symposium(NDSS 2018).Internet Soc,2018.
[17]CHEN X,LIU C,LI B,et al.Targeted backdoor attacks on deep learning systems using data poisoning[J].arXiv:1712.05526,2017.
[18]ZHAO S,MA X,ZHENG X,et al.Clean-label backdoor attacks on video recognition models[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:14443-14452.
[19]ZHANG Z K,PANG W G,XIE W J,et al.Review of deep lear-ning for real-time applications [J].Journal of Software,2019,31(9):2654-2677.
[20]RAKIN A S,HE Z,FAN D.Tbt:Targeted neural network attack with bit trojan[C]//Proceedings of the IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition.2020:13198-13207.
[21]LI Y,HUA J,WANG H,et al.DeepPayload:Black-box back-door attack on deep learning models through neural payload injection[C]//2021 IEEE/ACM 43rd International Conference on Software Engineering(ICSE).IEEE,2021:263-274.
[22]LIAO C,ZHONG H T,ANNA S,et al.Backdoor embedding in convolutional neural network models via invisible perturbation[J].arXiv:1808.10307,2018.
[23]LI S,XUE M,ZHAO B Z H,et al.Invisible backdoor attacks on deep neural networks via steganography and regularization[J].IEEE Transactions on Dependable and Secure Computing,2020,18(5):2088-2105.
[24]TAN T J L,SHOKRI R.Bypassing backdoor detection algorithms in deep learning[C]//2020 IEEE European Symposium on Security and Privacy(EuroS&P).IEEE,2020:175-183.
[25]ALI H,NEPAL S,KANHERE S S,et al.Has-nets:A heal and select mechanism to defend dnns against backdoor attacks for data collection scenarios[J].arXiv:2012.07474,2020.
[26]MA B,ZHAO C,WANG D,et al.DIHBA:Dynamic,invisible and high attack success rate boundary backdoor attack with low poison ratio[J].Computers & Security,2023,129:103212.
[27]ZHONG N,QIAN Z,ZHANG X.Imperceptible backdoor at-tack:From input space to feature representation[J].arXiv:2205.03190,2022.
[28]WANG T,YAO Y,XU F,et al.An Invisible Black-Box Backdoor Attack Through Frequency Domain[C]//Computer Vision－ECCV 2022:17th European Conference.Tel Aviv,Israel,2022:396-413.
[29]COSTALES R,MAO C,NORWITZ R,et al.Live trojan attacks on deep neural networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.2020:796-797.
[30]BARNI M,KALLAS K,TONDI B.A new backdoor attack in cnns by training set corruption without label poisoning[C]//2019 IEEE International Conference on Image Processing(ICIP).IEEE,2019:101-105.
[31]TURNER A,TSIPRAS D,MADRY A.Label-consistent backdoor attacks[J].arXiv:1912.02771,2019.
[32]SAHA A,SUBRAMANYA A,PIRSIAVASH H.Hidden trigger backdoor attacks[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:11957-11965.
[33]LIU Y,MA X,BAILEY J,et al.Reflection backdoor:A natural backdoor attack on deep neural networks[C]//Computer Vision－ECCV 2020:16th European Conference.Glasgow,UK,2020:182-199.
[34]LI H,WANG Y,XIE X,et al.Light can hack your face! black-box backdoor attack on face recognition systems[J].arXiv:2009.06996,2020.
[35]GAO Y,LI Y,ZHU L,et al.Not all samples are born equal:Towards effective clean-label backdoor attacks[J].Pattern Recognition,2023,139:109512.
[36]SALEM A,WEN R,BACKES M,et al.Dynamic backdoor attacks against machine learning models[C]//2022 IEEE 7th European Symposium on Security and Privacy(EuroS&P).IEEE,2022:703-718.
[37]NGUYEN T A,TRAN A.Input-aware dynamic backdoor attack[J].Advances in Neural Information Processing Systems,2020,33:3454-3464.
[38]LI Y,LI Y,WU B,et al.Invisible backdoor attack with sample-specific triggers[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:16463-16472.
[39]ZHANG J,DONGDONG C,HUANG Q,et al.Poison ink:Robust and invisible backdoor attack[J].IEEE Transactions on Image Processing,2022,31:5691-5705.
[40]CHENG S,LIU Y,MA S,et al.Deep feature space trojan attack of neural networks by controlled detoxification[C]//Procee-dings of the AAAI Conference on Artificial Intelligence.2021:1148-1156.
[41]ZHAO Z,CHEN X,XUAN Y,et al.DEFEAT:Deep Hidden Feature Backdoor Attacks by Imperceptible Perturbation and Latent Representation Constraints[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:15213-15222.
[42]QUIRING E,RIECK K.Backdooring and poisoning neural networks with image-scaling attacks[C]//2020 IEEE Security and Privacy Workshops(SPW).IEEE,2020:41-47.
[43]NGUYEN A,TRAN A.Wanet--imperceptible warping-basedbackdoor attack[J].arXiv:2102.10369,2021.
[44]SARKAR E,BENKRAOUDA H,MANIATAKOS M.Face-Hack:Triggering backdoored facial recognition systems using facial characteristics[J].arXiv:2006.11623,2020.
[45]ZHAO F,ZHOU L,ZHONG Q,et al.Natural Backdoor Attacks on Deep Neural Networks via Raindrops[J/OL].https://www.hindawi.com/journals/scn/2022/4593002/.
[46]BAGDASARYAN E,SHMATIKOV V.Blind backdoors in deep learning models[C]//Usenix Security.2021.
[47]DENG L.The mnist database of handwritten digit images formachine learning research [best of the web][J].IEEE Signal Processing Magazine,2012,29(6):141-142.
[48]NIU X,JIAO Y.An overview of perceptual hashing[J].ACTA ELECTONICA SINICA,2008,36(7):1405.
[49]ROZSA A,RUDD E M,BOULT T E.Adversarial diversity and hard positive generation[C]//Proceedings of the IEEE Confe-rence on Computer Vision and Pattern Recognition Workshops.2016:25-32.
[50]ZHANG R,ISOLA P,EFROS A A,et al.The unreasonable effectiveness of deep features as a perceptual metric[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:586-595.
[51]CHEN B,CARVALHO W,BARACALDO N,et al.Detectingbackdoor attacks on deep neural networks by activation clustering[J].arXiv:1811.03728,2018.
[52]LIU K,DOLAN-GAVITT B,GARG S.Fine-pruning:Defending against backdooring attacks on deep neural networks[C]//Research in Attacks,Intrusions,and Defenses:21st International Symposium,RAID 2018.2018:273-294.
[53]TRAN B,LI J,MADRY A.Spectral signatures in backdoor attacks[C]//NIPS’18.2018:8011-8021.
[54]GAO Y,XU C,WANG D,et al.Strip:A defence against trojan attacks on deep neural networks[C]//Proceedings of the 35th Annual Computer Security Applications Conference.2019:113-125.
[55]HONG S,CHANDRASEKARAN V,KAYA Y,et al.On the ef-fectiveness of mitigating data poisoning attacks with gradient shaping[J].arXiv:2002.11497,2020.
[56]KOLOURI S,SAHA A,PIRSIAVASH H,et al.Universal litmus patterns:Revealing backdoor attacks in cnns[C]//Procee-dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:301-310.
[57]MIRZA M,OSINDERO S.Conditional generative adversarialnets[J].arXiv:1411.1784,2014.
[58]LIU Y,XIE Y,SRIVASTAVA A.Neural trojans[C]//2017IEEE International Conference on Computer Design(ICCD).IEEE,2017:45-48.
[59]STALLKAMP J,SCHLIPSING M,SALMEN J,et al.Man vs.computer:Benchmarking machine learning algorithms for traffic sign recognition[J].Neural Networks,2012,32:323-332.
[60]SZEGEDY C,ZAREMBA W,SUTSKEVER I,et al.Intriguing properties of neural networks[J].arXiv:1312.6199,2014.
[61]GOODFELLOW I J,SHLENS J,SZEGEDY C.Explaining and harnessing adversarial examples[J].arXiv:1412.6572,2015.
[62]ROBERTS R D.Undersampled frequency shift ON-OFF keying(UFSOOK) for camera communications(CamCom)[C]//2013 22nd Wireless and Optical Communication Conference.IEEE,2013:645-648.
[63]YAO Y,LI H,ZHENG H,et al.Latent backdoor attacks on deep neural networks[C]//Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security.2019:2041-2055.
[64]LIU Y,LEE W C,TAO G,et al.Abs:Scanning neural networks for back-doors by artificial brain stimulation[C]//Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security.2019:1265-1282.
[65]WANG B,YAO Y,SHAN S,et al.Neural cleanse:Identifying and mitigating backdoor attacks in neural networks[C]//2019 IEEE Symposium on Security and Privacy(SP).IEEE,2019:707-723.
[66]BALUJA S.Hiding images in plain sight:Deep steganography[C]//NIPS’17.2017:2066-2076.
[67]TANCIK M,MILDENHALL B,NG R.Stegastamp:Invisiblehyperlinks in physical photographs[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:2117-2126.
[68]ZHU J,KAPLAN R,JOHNSON J,et al.Hidden:Hiding data with deep networks[C]//Proceedings of the European Confe-rence on Computer Vision(ECCV).2018:657-672.
[69]CHOU E,TRAMER F,PELLEGRINO G.Sentinet:Detectinglocalized universal attacks against deep learning systems[C]//2020 IEEE Security and Privacy Workshops(SPW).IEEE,2020:48-54.
[70]XIAO Q,CHEN Y,SHEN C,et al.Seeing is Not Believing:Camouflage Attacks on Image Scaling Algorithms[C]//USENIX Security Symposium.2019:443-460.
[71]DUCHON J.Splines minimizing rotation-invariant semi-norms in Sobolev spaces[C]//Constructive Theory of Functions of Several Variables.Berlin Heidelberg:Springer,1977:85-100.
[72]JADERBERG M,SIMONYAN K,ZISSERMAN A.Spatialtransformer networks[J].arXiv:1506.02025,2015.
[73]SARKAR E,ALKINDI Y,MANIATAKOS M.Backdoor sup-pression in neural networks using input fuzzing and majority voting[J].IEEE Design & Test,2020,37(2):103-110.
[74]VELDANDA A K,LIU K,TAN B,et al.Nnoculation:broad spectrum and targeted treatment of backdoored dnns[J].arXiv:2002.08313,2020.
[75]CHEN H,FU C,ZHAO J,et al.Proflip:Targeted trojan attack with progressive bit flips[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:7718-7727.
[76]AHMED S,MICHAEL B,AND YANG Z.Don’t trigger me! a triggerless backdoor attack against deep neural networks[J].arXiv:2010.03282,2020.
[77]TANG R X,DU M N,LIU N H,et al.An embarrassingly simple approach for trojan attack in deep neural networks[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.2020:218-228.
[78]GUO C,WU R H,KILIAN Q W.Trojannet:Embedding hidden trojan horse models in neural networks[J].arXiv:2002.10078,2020.
[79]WANG Y,CHEN K,HUANG S,et al.Stealthy and flexible trojan in deep learning framework[J].IEEE Transactions on Dependable and Secure Computing,2022,20(3):1789-1798.
[1] WANG Yao, LI Yi. Termination Analysis of Single Path Loop Programs Based on Iterative Trajectory Division [J]. Computer Science, 2023, 50(9): 108-116.
[2] YI Qiuhua, GAO Haoran, CHEN Xinqi, KONG Xiangjie. Human Mobility Pattern Prior Knowledge Based POI Recommendation [J]. Computer Science, 2023, 50(9): 139-144.
[3] LI Haiming, ZHU Zhiheng, LIU Lei, GUO Chenkai. Multi-task Graph-embedding Deep Prediction Model for Mobile App Rating Recommendation [J]. Computer Science, 2023, 50(9): 160-167.
[4] ZHU Ye, HAO Yingguang, WANG Hongyu. Deep Learning Based Salient Object Detection in Infrared Video [J]. Computer Science, 2023, 50(9): 227-234.
[5] YI Liu, GENG Xinyu, BAI Jing. Hierarchical Multi-label Text Classification Algorithm Based on Parallel Convolutional Network Information Fusion [J]. Computer Science, 2023, 50(9): 278-286.
[6] HENG Hongjun, MIAO Jing. Fusion of Semantic and Syntactic Graph Convolutional Networks for Joint Entity and Relation Extraction [J]. Computer Science, 2023, 50(9): 295-302.
[7] LI Ke, YANG Ling, ZHAO Yanbo, CHEN Yonglong, LUO Shouxi. EGCN-CeDML:A Distributed Machine Learning Framework for Vehicle Driving Behavior Prediction [J]. Computer Science, 2023, 50(9): 318-330.
[8] LU Yuhan, CHEN Liquan, WANG Yu, HU Zhiyuan. Efficient Encrypted Image Content Retrieval System Based on Secure CNN [J]. Computer Science, 2023, 50(9): 26-34.
[9] TANG Shaosai, SHEN Derong, KOU Yue, NIE Tiezheng. Link Prediction Model on Temporal Knowledge Graph Based on Bidirectionally Aggregating Neighborhoods and Global Aware [J]. Computer Science, 2023, 50(8): 177-183.
[10] MA Weiwei, ZHENG Qinhong, LIU Shanshan. Study and Evaluation of Spiking Neural Network Model Based on Bee Colony Optimization [J]. Computer Science, 2023, 50(8): 221-225.
[11] LI Qiaojun, ZHANG Wen, YANG Wei. Fusion Neural Network-based Method for Predicting LncRNA-disease Association [J]. Computer Science, 2023, 50(8): 226-232.
[12] XIE Tonglei, DENG Li, YOU Wenlong, LI Ruilong. Analysis and Prediction of Cloud VM CPU Load Based on EMPC-BCGRU [J]. Computer Science, 2023, 50(8): 243-250.
[13] WANG Yu, WANG Zuchao, PAN Rui. Survey of DGA Domain Name Detection Based on Character Feature [J]. Computer Science, 2023, 50(8): 251-259.
[14] LI Yang, LI Zhenhua, XIN Xianlong. Attack Economics Based Fraud Detection for MVNO [J]. Computer Science, 2023, 50(8): 260-270.
[15] ZHU Boyu, CHEN Xiao, SHA Letian, XIAO Fu. Two-layer IoT Device Classification Recognition Model Based on Traffic and Text Fingerprints [J]. Computer Science, 2023, 50(8): 304-313.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.