计算机科学 ›› 2022, Vol. 49 ›› Issue (6A): 178-183.doi: 10.11896/jsjkx.210500039

• 智能计算 • 上一篇    下一篇

基于动量的映射式梯度下降算法

吴子斌, 闫巧   

  1. 深圳大学计算机与软件学院 广东 深圳 518060
  • 出版日期:2022-06-10 发布日期:2022-06-08
  • 通讯作者: 闫巧(yanq@szu.edu.cn)
  • 作者简介:(695193423@qq.com)
  • 基金资助:
    国家自然科学基金面上项目(61976142)

Projected Gradient Descent Algorithm with Momentum

WU Zi-bin, YAN Qiao   

  1. College of Computer Science & Software Engineering,Shenzhen University,Shenzhen,Guangdong 518060,China
  • Online:2022-06-10 Published:2022-06-08
  • About author:WU Zi-bin,born in 1998.His main research interests include machine lear-ning and so on.
    YAN Qiao,born in 1972,Ph.D,professor,Ph.D supervisor,is a member of China Computer Federation.Her main research interests include network security,software-defined networking and machine learning.
  • Supported by:
    National Natural Science Foundation of China(61976142).

摘要: 近年来,深度学习已被广泛应用于计算机视觉问题中,并取得了卓越的成功。但研究人员发现神经网络容易受到添加微弱扰动的原始样本的干扰,导致模型给出一个错误的输出,这类输入样本称为“对抗样本”。目前已有一系列生成对抗样本的算法被提出。针对已有的对抗样本生成算法——映射式梯度下降算法(Projected Gradient Descent),提出了结合动量并采用新的损失函数的改进方法MPGDCW算法,以确保更新方向的稳定且避免不良局部最大值的出现,同时避免交叉熵损失函数可能出现的梯度消失情况。通过与包含3种架构4个鲁棒模型的实验,证实了所提MPGDCW算法具有更优的攻击效果和更强的攻击迁移性。

关键词: 对抗攻击, 卷积神经网络, 深度学习, 图像对抗样本

Abstract: In recent years,deep learning is widely used in the field of computer vision and has achieved outstanding success.However,the researchers found that the neural network is easily disturbed by adding subtle perturbations in the dataset,that can cause the model to give incorrect outputs.Such input examples are called “adversarial examples”.At present,a series of algorithms for generating adversarial examples have emerged.Based on the existing adversarial sample generation algorithm-projected gradient descent(PGD),this paper proposes an improved method-MPGDCW algorithm,which combines momentum and adopts a new loss function to ensure the stability of the update direction and avoid bad local maximums.At the same time,it can avoid the disappearance of the gradient by replacing the cross-entropy loss function.Experiments on 4 robust models containing 3 architecturesconfirm that the proposed MPGDCW algorithm has better attack effect and stronger transfer attack capacity.

Key words: Adversarial attacks, Convolutional neural network, Deep learning, Image adversarial examples

中图分类号: 

  • TP391.41
[1] SZEGEDY C,LIU W,JIA Y,et al.Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:1-9.
[2] MIKOLOV T,KARAFIÁT M,BURGET L,et al.Recurrentneural network based language model[C]//Eleventh Annual Conference of the International Speech Communication Association.2010.
[3] HINTON G,DENG L,YU D,et al.Deep neural networks foracoustic modeling in speech recognition:The shared views of four research groups[J].IEEE Signal Processing Magazine,2012,29(6):82-97.
[4] DEVLIN J,CHANG M W,LEE K,et al.Bert:Pre-training ofdeep bidirectional transformers for language understanding[J].arXiv:1810.04805,2018.
[5] KRIZHEVSKY A,SUTSKEVER I,HINTONG E.Imagenetclassification with deep convolutional neural networks[J].Advances in Neural Information Processing Systems,2012,25:1097-1105.
[6] REN S,HE K,GIRSHICK R,et al.Faster r-cnn:Towards real-time object detection with region proposal networks[J].arXiv:1506.01497,2015.
[7] LoNG J,SHELHAMER E,DARRELL T.Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:3431-3440.
[8] SZEGEDY C,ZAREMBA W,SUTSKEVERI,et al.Intriguing properties of neural networks[J].arXiv:1312.6199,2013.
[9] GOODFELLOW I J,SHLENS J,SZEGEDY C.Explaining and harnessing adversarial examples[J].arXiv:1412.6572,2014.
[10] KURAKIN A,GOODFELLOW I,BENGIO S.Adversarial examples in the physical world[J].arXiv:1607.02533,2016.
[11] MADRY A,MAKELOV A,SCHMIDT L,et al.Towards deep learning models resistant to adversarial attacks[J].arXiv:1706.06083,2017.
[12] CARLINI N,WAGNER D.Towards evaluating the robustness of neural networks[C]//2017 IEEE Symposium on Security and Privacy(sp).IEEE,2017:39-57.
[13] NIELSEN M A.Neural networks and deep learning(Vol.25)[M].San Francisco,CA:Determination Press,2015.
[14] POLYAK B T.Some methods of speeding up the convergence of iteration methods[J].Ussr Computational Mathematics and Mathematical Physics,1964,4(5):1-17.
[15] RUDER S.An overview of gradient descent optimization algorithms[J].arXiv:1609.04747,2016.
[16] DONG Y,LIAO F,PANG T,et al.Boosting adversarial attacks with momentum[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:9185-9193.
[17] CROCE F,HEIN M.Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks[C]//International Conference on Machine Learning.PMLR,2020:2206-2216.
[18] CROCE F,ANDRIUSHCHENKO M,SEHWAG V,et al.RobustBench:a standardized adversa-rial robustness benchmark[J].arXiv:2010.09670,2020.
[19] CROCE F,ANDRIUSHCHENKO M,SEHWAG V,et al.RobustBench/robustbench:RobustBench:a standardized adversa-rial robustness benchmark [EB/OL].https://github.com/RobustBench/robustbench.
[1] 周乐员, 张剑华, 袁甜甜, 陈胜勇.
多层注意力机制融合的序列到序列中国连续手语识别和翻译
Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion
计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026
[2] 徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺.
时序知识图谱表示学习
Temporal Knowledge Graph Representation Learning
计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[3] 饶志双, 贾真, 张凡, 李天瑞.
基于Key-Value关联记忆网络的知识图谱问答方法
Key-Value Relational Memory Networks for Question Answering over Knowledge Graph
计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[4] 汤凌韬, 王迪, 张鲁飞, 刘盛云.
基于安全多方计算和差分隐私的联邦学习方案
Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy
计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[5] 李宗民, 张玉鹏, 刘玉杰, 李华.
基于可变形图卷积的点云表征学习
Deformable Graph Convolutional Networks Based Point Cloud Representation Learning
计算机科学, 2022, 49(8): 273-278. https://doi.org/10.11896/jsjkx.210900023
[6] 王剑, 彭雨琦, 赵宇斐, 杨健.
基于深度学习的社交网络舆情信息抽取方法综述
Survey of Social Network Public Opinion Information Extraction Based on Deep Learning
计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[7] 郝志荣, 陈龙, 黄嘉成.
面向文本分类的类别区分式通用对抗攻击方法
Class Discriminative Universal Adversarial Attack for Text Classification
计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[8] 姜梦函, 李邵梅, 郑洪浩, 张建朋.
基于改进位置编码的谣言检测模型
Rumor Detection Model Based on Improved Position Embedding
计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[9] 陈泳全, 姜瑛.
基于卷积神经网络的APP用户行为分析方法
Analysis Method of APP User Behavior Based on Convolutional Neural Network
计算机科学, 2022, 49(8): 78-85. https://doi.org/10.11896/jsjkx.210700121
[10] 朱承璋, 黄嘉儿, 肖亚龙, 王晗, 邹北骥.
基于注意力机制的医学影像深度哈希检索算法
Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism
计算机科学, 2022, 49(8): 113-119. https://doi.org/10.11896/jsjkx.210700153
[11] 孙奇, 吉根林, 张杰.
基于非局部注意力生成对抗网络的视频异常事件检测方法
Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection
计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[12] 檀莹莹, 王俊丽, 张超波.
基于图卷积神经网络的文本分类方法研究综述
Review of Text Classification Methods Based on Graph Convolutional Network
计算机科学, 2022, 49(8): 205-216. https://doi.org/10.11896/jsjkx.210800064
[13] 侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木.
中文预训练模型研究进展
Advances in Chinese Pre-training Models
计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[14] 周慧, 施皓晨, 屠要峰, 黄圣君.
基于主动采样的深度鲁棒神经网络学习
Robust Deep Neural Network Learning Based on Active Sampling
计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
[15] 金方焱, 王秀利.
融合RACNN和BiLSTM的金融领域事件隐式因果关系抽取
Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM
计算机科学, 2022, 49(7): 179-186. https://doi.org/10.11896/jsjkx.210500190
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!