Computer Science ›› 2022, Vol. 49 ›› Issue (6A): 178-183.doi: 10.11896/jsjkx.210500039

• Intelligent Computing • Previous Articles     Next Articles

Projected Gradient Descent Algorithm with Momentum

WU Zi-bin, YAN Qiao   

  1. College of Computer Science & Software Engineering,Shenzhen University,Shenzhen,Guangdong 518060,China
  • Online:2022-06-10 Published:2022-06-08
  • About author:WU Zi-bin,born in 1998.His main research interests include machine lear-ning and so on.
    YAN Qiao,born in 1972,Ph.D,professor,Ph.D supervisor,is a member of China Computer Federation.Her main research interests include network security,software-defined networking and machine learning.
  • Supported by:
    National Natural Science Foundation of China(61976142).

Abstract: In recent years,deep learning is widely used in the field of computer vision and has achieved outstanding success.However,the researchers found that the neural network is easily disturbed by adding subtle perturbations in the dataset,that can cause the model to give incorrect outputs.Such input examples are called “adversarial examples”.At present,a series of algorithms for generating adversarial examples have emerged.Based on the existing adversarial sample generation algorithm-projected gradient descent(PGD),this paper proposes an improved method-MPGDCW algorithm,which combines momentum and adopts a new loss function to ensure the stability of the update direction and avoid bad local maximums.At the same time,it can avoid the disappearance of the gradient by replacing the cross-entropy loss function.Experiments on 4 robust models containing 3 architecturesconfirm that the proposed MPGDCW algorithm has better attack effect and stronger transfer attack capacity.

Key words: Adversarial attacks, Convolutional neural network, Deep learning, Image adversarial examples

CLC Number: 

  • TP391.41
[1] SZEGEDY C,LIU W,JIA Y,et al.Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:1-9.
[2] MIKOLOV T,KARAFIÁT M,BURGET L,et al.Recurrentneural network based language model[C]//Eleventh Annual Conference of the International Speech Communication Association.2010.
[3] HINTON G,DENG L,YU D,et al.Deep neural networks foracoustic modeling in speech recognition:The shared views of four research groups[J].IEEE Signal Processing Magazine,2012,29(6):82-97.
[4] DEVLIN J,CHANG M W,LEE K,et al.Bert:Pre-training ofdeep bidirectional transformers for language understanding[J].arXiv:1810.04805,2018.
[5] KRIZHEVSKY A,SUTSKEVER I,HINTONG E.Imagenetclassification with deep convolutional neural networks[J].Advances in Neural Information Processing Systems,2012,25:1097-1105.
[6] REN S,HE K,GIRSHICK R,et al.Faster r-cnn:Towards real-time object detection with region proposal networks[J].arXiv:1506.01497,2015.
[7] LoNG J,SHELHAMER E,DARRELL T.Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:3431-3440.
[8] SZEGEDY C,ZAREMBA W,SUTSKEVERI,et al.Intriguing properties of neural networks[J].arXiv:1312.6199,2013.
[9] GOODFELLOW I J,SHLENS J,SZEGEDY C.Explaining and harnessing adversarial examples[J].arXiv:1412.6572,2014.
[10] KURAKIN A,GOODFELLOW I,BENGIO S.Adversarial examples in the physical world[J].arXiv:1607.02533,2016.
[11] MADRY A,MAKELOV A,SCHMIDT L,et al.Towards deep learning models resistant to adversarial attacks[J].arXiv:1706.06083,2017.
[12] CARLINI N,WAGNER D.Towards evaluating the robustness of neural networks[C]//2017 IEEE Symposium on Security and Privacy(sp).IEEE,2017:39-57.
[13] NIELSEN M A.Neural networks and deep learning(Vol.25)[M].San Francisco,CA:Determination Press,2015.
[14] POLYAK B T.Some methods of speeding up the convergence of iteration methods[J].Ussr Computational Mathematics and Mathematical Physics,1964,4(5):1-17.
[15] RUDER S.An overview of gradient descent optimization algorithms[J].arXiv:1609.04747,2016.
[16] DONG Y,LIAO F,PANG T,et al.Boosting adversarial attacks with momentum[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:9185-9193.
[17] CROCE F,HEIN M.Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks[C]//International Conference on Machine Learning.PMLR,2020:2206-2216.
[18] CROCE F,ANDRIUSHCHENKO M,SEHWAG V,et al.RobustBench:a standardized adversa-rial robustness benchmark[J].arXiv:2010.09670,2020.
[19] CROCE F,ANDRIUSHCHENKO M,SEHWAG V,et al.RobustBench/robustbench:RobustBench:a standardized adversa-rial robustness benchmark [EB/OL].https://github.com/RobustBench/robustbench.
[1] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[2] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[3] ZHOU Le-yuan, ZHANG Jian-hua, YUAN Tian-tian, CHEN Sheng-yong. Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion [J]. Computer Science, 2022, 49(9): 155-161.
[4] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[5] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[6] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[7] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[8] CHEN Yong-quan, JIANG Ying. Analysis Method of APP User Behavior Based on Convolutional Neural Network [J]. Computer Science, 2022, 49(8): 78-85.
[9] ZHU Cheng-zhang, HUANG Jia-er, XIAO Ya-long, WANG Han, ZOU Bei-ji. Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism [J]. Computer Science, 2022, 49(8): 113-119.
[10] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[11] HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[12] DAI Zhao-xia, LI Jin-xin, ZHANG Xiang-dong, XU Xu, MEI Lin, ZHANG Liang. Super-resolution Reconstruction of MRI Based on DNGAN [J]. Computer Science, 2022, 49(7): 113-119.
[13] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[14] LIU Yue-hong, NIU Shao-hua, SHEN Xian-hao. Virtual Reality Video Intraframe Prediction Coding Based on Convolutional Neural Network [J]. Computer Science, 2022, 49(7): 127-131.
[15] XU Ming-ke, ZHANG Fan. Head Fusion:A Method to Improve Accuracy and Robustness of Speech Emotion Recognition [J]. Computer Science, 2022, 49(7): 132-141.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!