Computer Science ›› 2021, Vol. 48 ›› Issue (7): 33-39.doi: 10.11896/jsjkx.201200224

Special Issue: Artificial Intelligence Security

• Artificial Intelligence Security • Previous Articles     Next Articles

Differential Privacy Protection Machine Learning Method Based on Features Mapping

CHEN Tian-rong, LING Jie   

  1. School of Computer,Guangdong University of Technology,Guangdong 510006,China
  • Received:2020-12-25 Revised:2021-02-20 Online:2021-07-15 Published:2021-07-02
  • About author:CHEN Tian-rong,born in 1996,postgraduate.His main research interests include digital image processing and privacy protection.(1181113557@qq.com)
    LING Jie,born in 1964,Ph.D,professor.His main research interests include information security technology and intelligent video processing technology.
  • Supported by:
    Key Field R&D projects in Guangdong Province of China(2019B010139002) and Key Field R&D projects in Guangzhou(202007010004).

Abstract: The differential privacy algorithm in image classification improves the privacy protection capability of the machine learning model by adding noise,and at the same time easily causes the accuracy of the model classification to decrease.To solve the above problems,a differential privacy protection machine learning method based on features mapping is proposed.Thismethodcombines the pre-training neural network and shadow model training technology to map the feature vectors of the original data sample to the high-dimensional vector space in the form of differential vectors,so as to shorten the distance of the sample in the high-dimensional vector space to reduce the leakage of private information caused by model updates,and improve the privacy protection and classification capabilities of the machine learning model.The experimental results on the MNIST and CIFAR-10 datasets show that for the ε-differential privacy model with ε equal to 0.01 and 0.11,the classification accuracy is improved to 99% and 96%,respectively,indicating that compared with DP-SGD and many other commonly used differential privacy algorithms,the model trained by this method can maintain stronger classification capabilities at a lower privacy budget.And the success rate of reasoning attacks against this model on the two data sets is reduced to 10%,which is against inference attacks.Compared with the traditional CNN model of image classification,the defense capability of the CNN model is greatly improved.

Key words: Differential privacy, Image classification, Inference attack, Machine learning, Shadow model

CLC Number: 

  • TP391
[1]HA T,DANG T K,DANG T T,et al.Differential Privacy inDeep Learning:An Overview[C]//2019 International Confe-rence on Advanced Computing and Applications (ACOMP).Piscataway,NJ,USA:IEEE,2019:97-102.
[2]AHMED S,APRATIM B,MICHEAL B,et al.Updates-Leak:Data Set Inference and Reconstruction Attacks in Online Lear-ning[C]//29th USENIX Security Symposium.Online:USENIX Association,2019:1291-1308.
[3]SHOKRI R,STROATI M,SONG C Z,et al.Membership Infe-rence Attacks Against Machine Learning Models[C]//2017 38th IEEE Symposium on Security and Privacy (SP).Los Alamitos,CA,USA:IEEE Computer Society,2017:3-18.
[4]DWORK C,KENTHAPADI K,MCSHERRY F,et al.Our data,ourselves:privacy via distributed noise generation[C]//24th Annual International Conference on the Theory and Applications of Cryptographic Techniques Advances in Cryptology(EUROCRYPT 2006).Berlin,Germany:IEEE Computer Socie-ty,2006:486-503.
[5]ABADI M,MCMAHANH B,CHU A,et al.Deep learning with differential privacy[C]//Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security(CCS 2016).Vienna,Austria:Association for Computing Machinery,2016:308-318.
[6]XIE L Y,LIN K X,WANG S,et al.Differentially Private Gene-rative Adversarial Network[J/OL].http://arxiv.org/abs/1802.06739,2020-5-13.
[7]PHAN N,WANG Y,WU X,et al.Differential privacy preservation for deep auto-encoders:An application of human behavior prediction[C]//30th AAAI Conference on Artificial Intelligence(AAAI 2016).Phoenix,AZ,United states:AAAI press,2016:1309-1316.
[8]PHAN N,WU X,HU H,et al.Adaptive Laplace mechanism:differential privacy preservation in deep learning[C]//2017 IEEE International Conference on Data Mining (ICDM).Los Alamitos,CA,USA:IEEE Computer Society,2017:385-394.
[9]PAPERNOT N,GOODFELLOW I,ABADI M,et al.Semi-supervised knowledge transfer for deep learning from private training data[C]//5th International Conference on Learning Representations(ICLR 2017).Conference Track Proceedings.Toulon,France:ICLR,2017:1024-1040.
[10]GANJU K,WANG Q,YANG W,et al.Property inference attacks on fully connected neural networks using permutation invariant representations[C]//Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security(CCS 2018).United States:Association for Computing Machi-nery,2018:619-633.
[11]JOON O S,BERNT S,MARIO F.Towards Reverse-Enginee-ring Black-Box Neural Networks[J].Springer Verlag,2017,11700(2017):121-144.
[12]SALEM A,YANG Z,HUMBERT M,et al.ML-Leaks:Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models[J/OL].http://arxiv.org/abs/1806.01246,2018-12-14.
[13]SHOKRI R,STRONATI M,SONG C,et al.Membership Infe-rence Attacks Against Machine Learning Models[C]//2017 38th IEEE Symposium on Security and Privacy (SP).Los Alamitos,CA,USA:IEEE Computer Society,2017:3-18.
[14]WANG B,GONG N.Stealing Hyperparameters in MachineLearning[C]//2018 IEEE Symposium on Security and Privacy (SP).Los Alamitos,CA,USA:IEEE Computer Society,2018:36-52.
[15]PHAN N,WU X,DOU D.Preserving differential privacy in convolutional deep belief networks[J].MACH LEARN,2017,106:1681-1704.
[16]GONG M,PAN K,XIE Y,et al.Preserving differential privacy in deep neural networks with relevance-based adaptive noise imposition[J].Neural Networks,2020,125:131-141.
[17]DONG J S,ROTH A,SU W J,et al.Gaussian Differential Privacy [J/OL].http://arxiv.org/abs/1905.02383,2019-10-08.
[1] LENG Dian-dian, DU Peng, CHEN Jian-ting, XIANG Yang. Automated Container Terminal Oriented Travel Time Estimation of AGV [J]. Computer Science, 2022, 49(9): 208-214.
[2] NING Han-yang, MA Miao, YANG Bo, LIU Shi-chang. Research Progress and Analysis on Intelligent Cryptology [J]. Computer Science, 2022, 49(9): 288-296.
[3] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[4] HE Qiang, YIN Zhen-yu, HUANG Min, WANG Xing-wei, WANG Yuan-tian, CUI Shuo, ZHAO Yong. Survey of Influence Analysis of Evolutionary Network Based on Big Data [J]. Computer Science, 2022, 49(8): 1-11.
[5] WU Hong-xin, HAN Meng, CHEN Zhi-qiang, ZHANG Xi-long, LI Mu-hang. Survey of Multi-label Classification Based on Supervised and Semi-supervised Learning [J]. Computer Science, 2022, 49(8): 12-25.
[6] LI Yao, LI Tao, LI Qi-fan, LIANG Jia-rui, Ibegbu Nnamdi JULIAN, CHEN Jun-jie, GUO Hao. Construction and Multi-feature Fusion Classification Research Based on Multi-scale Sparse Brain Functional Hyper-network [J]. Computer Science, 2022, 49(8): 257-266.
[7] ZHANG Guang-hua, GAO Tian-jiao, CHEN Zhen-guo, YU Nai-wen. Study on Malware Classification Based on N-Gram Static Analysis Technology [J]. Computer Science, 2022, 49(8): 336-343.
[8] CHEN Ming-xin, ZHANG Jun-bo, LI Tian-rui. Survey on Attacks and Defenses in Federated Learning [J]. Computer Science, 2022, 49(7): 310-323.
[9] HUANG Jue, ZHOU Chun-lai. Frequency Feature Extraction Based on Localized Differential Privacy [J]. Computer Science, 2022, 49(7): 350-356.
[10] DU Li-jun, TANG Xi-lu, ZHOU Jiao, CHEN Yu-lan, CHENG Jian. Alzheimer's Disease Classification Method Based on Attention Mechanism and Multi-task Learning [J]. Computer Science, 2022, 49(6A): 60-65.
[11] LI Ya-ru, ZHANG Yu-lai, WANG Jia-chen. Survey on Bayesian Optimization Methods for Hyper-parameter Tuning [J]. Computer Science, 2022, 49(6A): 86-92.
[12] ZHAO Lu, YUAN Li-ming, HAO Kun. Review of Multi-instance Learning Algorithms [J]. Computer Science, 2022, 49(6A): 93-99.
[13] YANG Jian-nan, ZHANG Fan. Classification Method for Small Crops Combining Dual Attention Mechanisms and Hierarchical Network Structure [J]. Computer Science, 2022, 49(6A): 353-357.
[14] XIAO Zhi-hong, HAN Ye-tong, ZOU Yong-pan. Study on Activity Recognition Based on Multi-source Data and Logical Reasoning [J]. Computer Science, 2022, 49(6A): 397-406.
[15] YAO Ye, ZHU Yi-an, QIAN Liang, JIA Yao, ZHANG Li-xiang, LIU Rui-liang. Android Malware Detection Method Based on Heterogeneous Model Fusion [J]. Computer Science, 2022, 49(6A): 508-515.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!