计算机科学 ›› 2021, Vol. 48 ›› Issue (10): 258-265.doi: 10.11896/jsjkx.200800222

• 信息安全 • 上一篇    下一篇

基于特征变换的图像检索对抗防御

徐行, 孙嘉良, 汪政, 杨阳   

  1. 电子科技大学计算机科学与工程学院 成都611731
  • 收稿日期:2020-08-30 修回日期:2021-03-05 出版日期:2021-10-15 发布日期:2021-10-18
  • 通讯作者: 徐行(interxuxing@hotmail.com)

Feature Transformation for Defending Adversarial Attack on Image Retrieval

XU Xing, SUN Jia-liang, WANG Zheng, YANG Yang   

  1. School of Computer Science and Engineering,University of Electronic Science and Technology of China,Chengdu 611731,China
  • Received:2020-08-30 Revised:2021-03-05 Online:2021-10-15 Published:2021-10-18
  • About author:XU Xing,born in 1988,Ph.D,associate professor,is a member of China Computer Federation.His main research interests include multimedia information processing and security,cross-media analysis and computer vision.

摘要: 对抗攻击在图像分类中较早被研究,目的是产生可以误导神经网络预测的不可察觉的扰动。最近,图像检索中的对抗攻击也被广泛探索,研究结果表明最先进的基于深度神经网络的图像检索模型同样容易受到干扰,从而将不相关的图像返回。文中首次尝试研究无需训练的图像检索模型的对抗防御方法,根据图像基本特征因素对输入图像进行变换,以在预测阶段消除对抗攻击的影响。所提方法探索了4种图像特征变换方案,即调整大小、填充、总方差最小化和图像拼接,这些都是在查询图像被送入检索模型之前对其执行的。文中提出的防御方法具有以下优点:1)不需要微调和增量训练过程;2)仅需极少的额外计算;3)多个方案可以灵活集成。大量实验的结果表明,提出的变换策略在防御现有的针对主流图像检索模型的对抗攻击方面是非常有效的。

关键词: 对抗防御, 对抗攻击, 深度学习, 图像变换, 图像检索

Abstract: The adversarial attack is firstly studied in image classification to generate imperceptible perturbations that can mislead the prediction of a convolutional neural network.Recently,it has also been extensively explored in image retrieval and shows that the popular image retrieval models are undoubtedly vulnerable to return irrelevant images to the query image with small perturbations.In particular,landmark image retrieval is a research hotspot of image retrieval as an explosive volume of landmark images are uploaded on the Internet by people using various smart devices when taking tours in cities.This paper makes the first trail to investigate the defending approach against adversarial attacks on city landmark image retrieval models without training.Specifica-lly,we propose to perform image feature transformation at inference time to eliminate the adversarial effects based on the basic image features.Our method explores four feature transformation schemes:resize,padding,total variance minimization and image quilting,which are performed on a query image before feeding it to a retrieval model.Our defense method has the following advantages:1) no fine-tuning and incremental training procedure is required,2) very few additional computations and 3) flexible ensembles of multiple schemes.Extensive experiments show that the proposed transformation strategies are advanced at defending the existing adversarial attacks performed on the state-of-the-art city landmark image retrieval models.

Key words: Adversarial attack, Adversarial defence, Deep learning, Feature transformation, Image retrieval

中图分类号: 

  • TP37
[1]FILIP R,AHMET I,GIORGOS T,et al.Revisiting Oxford and Paris:Large-Scale Image Retrieval Benchmarking [C]//IEEE International Conference on Computer Vision and Pattern Re-cognition (CVPR).2018:5706-5715.
[2]RADENOVIC F,TOLIAS G,AND O C.Fine-Tuning CNNImage Retrieval with No Human Annotation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,41(7):1655-1668.
[3]CHRISTIAN S,WOJCIECH Z,ILYA S,et al.Intriguing pro-perties of neural networks[C]//International Conference on Learning Representation.2014.
[4]LIU Z R,ZHAO Z Y,MARTHA L.Who's Afraid of Adversa-rial Queries? The Impact of Image Modifications on Content-based Image Retrieval[C]//International Conference on Multimedia Retrieval.2019:578-586.
[5]GIORGOS T,FILIP R,ONDREJ C.Targeted Mismatch Adversarial Attack:Query With a Flower to Retrieve the Tower[C]//IEEE/CVF International Conference on Computer Vision.2019:5036-5045.
[6]LI J,JI R,LIU H,et al.Universal perturbation attack against image retrieval[C]//Proceedings of the IEEE International Conference on Computer Vision.2019:4899-4908.
[7]JOSEF S,ANDREW Z.Video Google:A Text Retrieval Ap-proach to Object Matching in Videos[C]//IEEE International Conference on Computer Vision.2003:1470-1477.
[8]JAMES P,ONDREJ C,MICHAEL I,et al.Object retrieval with large vocabularies and fast spatial matching[C]//IEEE International Conference on Computer Vision and Pattern Recognition.2007:1533-1540.
[9]ONDREJ C,JAMES P,JOSEF S,et al.Total Recall:Automatic Query Expansion with a Generative Feature Model for Object Retrieval[C]//IEEE International Conference on Computer Vision.2007:1-8.
[10]HERVÉ J,MATTHIJS D,CORDELIA S.Hamming Embedding and Weak Geometric Consistency for Large Scale Image Search[C]//European Conference on Computer Vision.2008:304-317.
[11]ZHANG S S,ZUO X,LIU J W.The Problem of the Adversarial Examples in Deep Learning[J].Chinese Journal of Computers,2019,42(8):1886-1904.
[12]IAN G,JONATHON S,CHRISTIAN S.Explaining and Harnessing Adversarial Examples[C]//International Conference on Learning Representations.2015:1-12.
[13]HYEONWOO N,ANDRE A,JACK S,et al.Large-Scale Image Retrieval With Attentive Deep Local Features[C]//The IEEE International Conference on Computer Vision (ICCV).2017:567-575.
[14]FROSSARD P,MOSSAVI-DEZFOOLI S M,FAWZI A,et al.DeepFool:A Simple and Accurate Method to Fool Deep Neural Networks[C]//IEEE Conference on Computer Vision and Pattern Recognition,CVPR.2016:2574-2582.
[15]NICHOLAS C,DAVID A,WAGNE R.Towards Evaluating the Robustness of Neural Networks[C]//IEEE Symposium on Security and Privacy.2017:1-16.
[16]NICOLAS P,PATRICK D,MCDANIEL P,et al.PracticalBlack-Box Attacks against Machine Learning[C]//Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security.2017:506-519.
[17]CHEN P Y,ZHANG H,YASH S,et al.ZOO:Zeroth Order Optimization Based Black-box Attacks to Deep Neural Networks without Training Substitute Models[C]//Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security.2017:15-26.
[18]MOOSAVI-DEZFOOLI S M,FAWZI A,FAWII O,et al.Universal Adversarial Perturbations[C]//The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).2017:445-452.
[19] ZHENG Z D,ZHENG L,HU Z L,et al.Open Set Adversarial Examples[OL].CoRR abs/1809.02681.https://www.resear-chgate.net/publication/327570780_Open_Set_Adversarial_Examples.
[20]HE Y Z,HU X B,HE J W,et al.Privacy and Security Issues in Machine Learning Systems:A Survey[J].Journal of Computer Research and Development,2019,56(10):2049-2070.
[21] YUAN X Y,HE P,ZHU Q L,et al.Adversarial Examples:Attacks and Defenses for Deep Learning[J].IEEE Transactions on Neural Networks and Learning Systems,2019,30(9):2805-2824.
[22]CHUAN G,MAYANK R,MOUSTAPHA C,et al.CounteringAdversarial Images using Input Transformations[C]//International Conference on Learning Representations.2018:1-12.
[23]JAN H M,TIM G,VOLKER F,et al.On Detecting Adversarial Perturbations[C]//International Conference on Learning Representations.2017:1-12.
[24]MADRY A,ALEKSANDAR M,LUDWIG S,et al.TowardsDeep Learning Models Resistant to Adversarial Attacks[C]//International Conference on Learning Representations.2018:1-10.
[25]GUY K,CLARK W,BARRETT C,et al.Towards Proving the Adversarial Robustness of Deep Neural Networks[C]//Proceedings First Workshop on Formal Verification of Autonomous Vehicles.2017:19-26.
[26]KONG R,CAI J C,HUANG G.Defense to Adversarial Attack with Generative Adversarial Network [J/OL].Acta Automatica Sinica.https://doi.org/10.16383/j.aas.c200033.
[27]DIEDERIK K,JIMMY B.ADAM:a method for stochastic optimization[C]//International Conference on Learning Representations.2015:1-10.
[28]ALEX K,ILYA S,GEOFFREY E H.ImageNet Classification with Deep Convolutional Neural Networks[C]//Neural Information Processing Systems(NIPS).2012:1106-1114.
[29]LEONID R,STANLEY O,EMAD F.Nonlinear total variation based noise removal algorithms[J].Physica D:Nonlinear Phenomena,1992,60(1/2/3/4):259-268.
[30]ALEXEI A,EFRO S,WILLIAM F.Image quilting for texture synthesis and transfer[C]//Special Interest Group on Computer Graphics and Interactive(SIGGRAPH).2001:341-346.
[31]YURI B,OLGA V,RAMIN Z.Fast approximate energy minimization via graph cuts[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2001,23(11):1222-1239.
[32]ALI S R,JOSEPHINE S,ATSUTO M,et al.Visual Instance Retrieval with Deep Convolutional Networks[C]//International Conference on Learning Representations.2016:1-10.
[33]BABENKO A,LEMPITSKY V.Aggregating Deep Convolu-tional Features for Image Retrieval[C]//International Con-ference on Computer Vision.2015:1246-1254.
[34]YANNIS K,CLAYTON M,SIMON O.Cross-DimensionalWeighting for Aggregated Deep Convolutional Features[C]//European Conference on Computer Vision Workshops.2016:685-701.
[35]DENG J,WEI D,RICHARD S,et al.Imagenet:a large-scalehierarchical image database[C]//IEEE Conference on Computer Vision and Pattern Recognition.2009:1573-1580.
[36]SCHONBERGER L,FILIP R,ONDREJ C,et al.From single image query to detailed 3d reconstruction[C]//Computer Vision and Pattern Recognition.2015:485-492.
[37]ATHALYE A,ENGSTROM L,ILYAS A,et al.Synthesizing robust adversarial examples[C]//International Conference on Machine Learning.PMLR,2018:284-293.
[1] 饶志双, 贾真, 张凡, 李天瑞.
基于Key-Value关联记忆网络的知识图谱问答方法
Key-Value Relational Memory Networks for Question Answering over Knowledge Graph
计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[2] 汤凌韬, 王迪, 张鲁飞, 刘盛云.
基于安全多方计算和差分隐私的联邦学习方案
Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy
计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[3] 徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺.
时序知识图谱表示学习
Temporal Knowledge Graph Representation Learning
计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[4] 王剑, 彭雨琦, 赵宇斐, 杨健.
基于深度学习的社交网络舆情信息抽取方法综述
Survey of Social Network Public Opinion Information Extraction Based on Deep Learning
计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[5] 郝志荣, 陈龙, 黄嘉成.
面向文本分类的类别区分式通用对抗攻击方法
Class Discriminative Universal Adversarial Attack for Text Classification
计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[6] 姜梦函, 李邵梅, 郑洪浩, 张建朋.
基于改进位置编码的谣言检测模型
Rumor Detection Model Based on Improved Position Embedding
计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[7] 孙奇, 吉根林, 张杰.
基于非局部注意力生成对抗网络的视频异常事件检测方法
Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection
计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[8] 胡艳羽, 赵龙, 董祥军.
一种用于癌症分类的两阶段深度特征选择提取算法
Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification
计算机科学, 2022, 49(7): 73-78. https://doi.org/10.11896/jsjkx.210500092
[9] 程成, 降爱莲.
基于多路径特征提取的实时语义分割方法
Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction
计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[10] 侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木.
中文预训练模型研究进展
Advances in Chinese Pre-training Models
计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[11] 周慧, 施皓晨, 屠要峰, 黄圣君.
基于主动采样的深度鲁棒神经网络学习
Robust Deep Neural Network Learning Based on Active Sampling
计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
[12] 苏丹宁, 曹桂涛, 王燕楠, 王宏, 任赫.
小样本雷达辐射源识别的深度学习方法综述
Survey of Deep Learning for Radar Emitter Identification Based on Small Sample
计算机科学, 2022, 49(7): 226-235. https://doi.org/10.11896/jsjkx.210600138
[13] 王君锋, 刘凡, 杨赛, 吕坦悦, 陈峙宇, 许峰.
基于多源迁移学习的大坝裂缝检测
Dam Crack Detection Based on Multi-source Transfer Learning
计算机科学, 2022, 49(6A): 319-324. https://doi.org/10.11896/jsjkx.210500124
[14] 楚玉春, 龚航, 王学芳, 刘培顺.
基于YOLOv4的目标检测知识蒸馏算法研究
Study on Knowledge Distillation of Target Detection Algorithm Based on YOLOv4
计算机科学, 2022, 49(6A): 337-344. https://doi.org/10.11896/jsjkx.210600204
[15] 刘伟业, 鲁慧民, 李玉鹏, 马宁.
指静脉识别技术研究综述
Survey on Finger Vein Recognition Research
计算机科学, 2022, 49(6A): 1-11. https://doi.org/10.11896/jsjkx.210400056
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!