Computer Science

Computer Science ›› 2023, Vol. 50 ›› Issue (5): 372-381.doi: 10.11896/jsjkx.220300239

• Information Security • Previous Articles     Next Articles

Neural Network Model Training Method Based on Homomorphic Encryption

ZHAO Min1,2,3, TIAN Youliang1,2,3, XIONG Jinbo1,2,4, BI Renwan4, XIE Hongtao5   

  1. 1 State Key Laboratory of Public Big Date,Guizhou University,Guiyang 550025,China
    2 College of Computer Science and Technology,Guizhou University,Guiyang,550025,China
    3 Institute of Cryptography & Data Security,Guizhou University,Guiyang 550025,China
    4 College of Computer and Cyber Security,Fujian Normal University,Fuzhou 350117,China
    5 School of Information Science and Technology,University of Science and Technology of China,Hefei 230000,China
  • Received:2022-03-25 Revised:2022-12-30 Online:2023-05-15 Published:2023-05-06
  • About author:ZHAO Min,born in 1995,postgraduate.Her main research interests include secure machine learning and privacy protection.
    XIONG Jinbo,born in 1981,Ph.D,professor,Ph.D supervisor.His main research interests include secure deep learning,mobile crowdsensing security and privacy protection.
  • Supported by:
    National Key Research and Development Program of China(2021YFB3101100),National Natural Science Foundation of China(62272123,62272102),Project of High-level Innovative Talents of Guizhou Province([2020]6008),Science and Technology Program of Guiyang([2021]1-5,[2022]2-4) and Science and Technology Program of Guizhou Province([2020]5017,[2022]065).

PDF (PC)

519

Abstract: Aiming at the problem of data privacy leakage in cloud environment and insufficient accuracy in the privacy-preserving neural network based on homomorphic encryption,a privacy-preserving neural network training scheme(PPNT) is proposed for collaborative dual cloud servers,to achieve the goal of data transmission,computing security and model parameter under the collaborative training process of dual cloud servers.Firstly,in order to avoid using polynomial approximation method to realize nonlinear functions such as exponent and comparison,and improve the calculation accuracy of nonlinear function,a series of secure computing protocols are designed based on Paillier partially homomorphic encryption technology and additive secret sharing scheme.Furthermore,corresponding secure computing protocols of full connection layer,activation layer,softmax layer and back propagation in neural network are constructed to realize PPNT based on the designed secure computing protocols.Finally,theoretical and security analysis guarantees the correctness and security of PPNT.The actual performance results show that compared with the dual server scheme--privacy protection machine learning as a service(PPMLaaS),the model accuracy of PPNT improves by 1.7%,and supports the client offline in the process of secure computing.

Key words: Paillier partially homomorphic encryption, Additive secret sharing, Secure computing protocol, Privacy-preserving, Model training

CLC Number: 

  • TP309.2
[1]MA Z,LIU Y,LIU X,et al.Lightweight privacy-preserving ensemble classification for face recognition[J].IEEE Internet of Things Journal,2019,6(3):5778-5790.
[2]LUO X,LI L,WAN H,et al.Phone keypad voice recognition:an integrated experiment for digital signal processing education[C]//Proceedings of the 2020 IEEE Frontiers in Education Conference.Piscataway:IEEE Press,2020:1-4.
[3]LI Z Y,GUI X L,GU Y J,et al.Survey on homomorphic encryption algorithm and its application in the privacy-preserving for cloud computing[J].Journal of Software,2018,29(7):1830-1851.
[4]TAN Z W,ZHANG L F.Survey on privacy preserving techniques for machine learning[J].Journal of Software,2020,31(7):2127-2156.
[5]GILAD-BACHRACH R,DOWLIN N,LAINE K,et al.Cryp-tonets:applying neural networks to encrypted data with high throughput and accuracy[C]//International Conference on Machine Learning.New York:ACM Press,2016:201-210.
[6]HESAMIFARD E,TAKABI H,GHASEMI M.Cryptodl:Deep neural networks over encrypted data[J].arXiv:1711.05189,2017.
[7]CHOU E,BEAL J,LEVY D,et al.Faster cryptonets:leveraging sparsity for real-world encrypted inference[J].arXiv:1811.09953,2018.
[8]CHABANNE H,DE W A,MILGRAM J,et al.Privacy-preserving classification on deep neural network[J／OL].Cryptology ePrint Archive,2017,1-35.http://eprint.iacr.org/2017/035.
[9]JUVEKAR C,VALKUNTANATHAN V,CHANDRAKASAN A.{GAZELLE}:A low latency framework for secure neural network inference[C]//27th USENIX Security Symposium({USENIX} Security 18).Berkeley:USENIX Association,2018:1651-1669.
[10]BADAWI A,CHAO J,JIE L,et al.Towards the alexnet mo-ment for homomorphic encryption:hcnn,the first homomorphic cnn on encrypted data with gpus[J].IEEE Transactions on Emerging Topics in Computing,2021,9(3):1330-1343.
[11]HAN K,HONG S,CHEON J H,et al.Logistic regression onhomomorphic encrypted data at scale[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Menlo Park:AAAI Press,2019:9466-9471.
[12]BRAKERSKI Z,GENTRY C,VAIKUNTANATHAN V.(Le-veled) fully homomorphic encryption without bootstrapping[J].ACM Transactions on Computation Theory(TOCT),2014,6(3):1-36.
[13]ZHANG Q,WANG C,WU H,et al.GELU-Net:a globally encrypted,locally unencrypted deep neural network for privacy-preserved learning[C]//Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence.Stockholm:IJCAI.2018:3933-3939.
[14]BOURSE F,MINELLI M,MINIHOLD M,et al.Fast homomorphic evaluation of deep discretized neural networks[C]//Annual International Cryptology Conference.Berlin:Springer,2018:483-512.
[15]CHILLOTTI I,GAMA N,GEORGIEVA M,et al.Faster fully homomorphic encryption:Bootstrapping in less than 0.1 seconds[C]//International Conference on the Theory and Application of Cryptology and Information Security.Berlin:Springer,2016:3-33.
[16]HESAMIFARD E,TAKABI H,GHASEMI M,et al.Privacy-preserving machine learning in cloud[C]//Proceedings of the 2017 on Cloud Computing Security Workshop.New York:ACM Press,2017:39-43.
[17]LOU Q,FENG B,CHARLES F G,et al.Glyph:fast and accurately training deep neural networks on encrypted data[J/OL].Advances in Neural Information Processing Systems,2020,33:9193-9202.https://proceedings.neurips.cc/paper/2020/hash/685ac8cadc1be5ac98da9556bc1c8d9e-Abstract.html.
[18]PAILLIER P.Public-key cryptosystems based on composite degree residuosity classes[C]//Proceedings of the International Conference on the Theory and Dpplications of Cryptographic Techniques.Berlin:Springer,1999:223-238.
[19]SHAMIR A.How to share a secret[J].Communications of the ACM,1979,22(11):612-613.
[20]LIU Y,MA Z,LIU X,et al.Privacy-preserving object detection for medical images with faster R-CNN[J／OL].IEEE Transactions on Information Forensics and Security,2022,17:69-84.https://doi.org/10.1109/TIFS.2019.2946476.
[21]XIONG J B,BI R W,TIAN Y L,et al.Towards lightweight,privacy-preserving cooperative object classification for connected autonomous vehicles[J].IEEE Internet of Things Journal,2021,9(4):2787-2801.
[22]HUANG K,LIU X,FU S,et al.A lightweight privacy-preserving CNN feature extraction framework for mobile sensing[J].IEEE Transactions on Dependable and Secure Computing,2019,18(3):1441-1455.
[23]XIONG J B,ZHOU Y J,BI R W,et al.Towards edge-collaborative,lightweight and privacy-preserving classification framework[J].Journal on Communications,2022,43(1):127-137.
[24]MA Z,LIU Y,LIU X,et al.Privacy-preserving outsourcedspeech recognition for smart IoT devices[J].IEEE Internet of Things Journal,2019,6(5):8406-8420.
[25]BI R W,CHEN Q X,XIONG J B,et al.Design method of secure computing protocol for deep neural network[J].Chinese Journal of Network and Information Security,2020,6(4):130-139.
[26]WAGH S,TOPLE S,BENHAMOUDA F,et al.Falcon:honest-majority maliciously secure framework for private deep learning[J].Privacy Enhancing Technologies,2021,2021(1):188-208.
[27]BOGDANOV D,NIITSOO M,TOFT T,et al.High-perfor-mance secure multi-party computation for data mining applications[J].International Journal of Information Security,2012,11(6):403-418.
[28]XIONG J,BI R,ZHAO M,et al.Edge-assisted privacy-preserving raw data sharing framework for connected autonomous vehicles[J].IEEE Wireless Communications,2020,27(3):24-30.
[29]XIONG J B,BI R W,CHEN Q X,et al.Towards edge-collaborative,lightweight and secure region proposal network[J].Journal on Communications,2020,41(10):188-201.
[30]HE K,ZHANG X,REN S,et al.Delving deep into rectifiers:surpassing human-level performance on imagenet classification[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society,2015:1026-1034.
[31]MOHASSEL P,ZHANG Y.Secureml:a system for scalable privacy-preserving machine learning[C]//2017 IEEE Symposium on Security and Privacy(SP).Piscataway:IEEE Press,2017:19-38.
[32]HESAMIFARD E,TAKABI H,GHASEMI M,et al.Privacy-preserving machine learning as a service[J].Proceedings on Privacy Enhancing Technologies,2018,2018(3):123-142.
[33]LIU J,JUUTI M,LU Y,et al.Oblivious neural network predictions via minionn transformations[C]//Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security.New York:ACM Press,2017:619-631.
[1] CHEN Depeng, LIU Xiao, CUI Jie, HE Daojing. Survey of Membership Inference Attacks for Machine Learning [J]. Computer Science, 2023, 50(1): 302-317.
[2] LYU You, WU Wen-yuan. Privacy-preserving Linear Regression Scheme and Its Application [J]. Computer Science, 2022, 49(9): 318-325.
[3] LI Li, HE Xin, HAN Zhi-jie. Review of Privacy-preserving Mechanisms in Crowdsensing [J]. Computer Science, 2022, 49(5): 303-310.
[4] QU Xiang-mou, WU Ying-bo, JIANG Xiao-ling. Federated Data Augmentation Algorithm for Non-independent and Identical Distributed Data [J]. Computer Science, 2022, 49(12): 33-39.
[5] JI Yan, DAI Hua, JIANG Ying-ying, YANG Geng, Yi Xun. Parallel Multi-keyword Top-k Search Scheme over Encrypted Data in Hybrid Clouds [J]. Computer Science, 2021, 48(5): 320-327.
[6] LI Yan-bin, LIU Yu, LI Mu-zhou, WU Ren-tao, WANG Peng-da. Participant-adaptive Variant of MASCOT [J]. Computer Science, 2020, 47(11A): 380-387.
[7] CAI Wei, BAI Guang-wei, SHEN Hang, CHENG Zhao-wei, ZHANG Hui-li. Reinforcement Learning Based Win-Win Game for Mobile Crowdsensing [J]. Computer Science, 2020, 47(10): 41-47.
[8] LIU Meng-jun, LIU Shu-bo and DING Yong-gang. 0-1 Code Based Privacy-preserving Data Value Matching in Participatory Sensing [J]. Computer Science, 2018, 45(3): 131-137.
[9] ZHANG Yan-ping and LING Jie. Improved Algorithm for Privacy-preserving Association Rules Mining on Horizontally Distributed Databases [J]. Computer Science, 2017, 44(8): 157-161.
[10] ZHANG Xiao-lin, ZHANG Chen, ZHANG Wen-chao, ZHANG Huan-xiang and YU Fang-ming. D-VSSP:Distributed Social Network Privacy Preserving Algorithm [J]. Computer Science, 2017, 44(2): 93-97.
[11] CHEN Chao-qun and LI Zhi-hua. Privacy-preserving Oriented Ciphertext Retrieval Algorithm [J]. Computer Science, 2016, 43(Z11): 346-351.
[12] YANG Zhao-huan, LIU Shu-bo, LI Yong-kai and CAI Chao-hui. Symptoms Privacy-preserving Matching Protocol for m-Healthcare Social Network [J]. Computer Science, 2015, 42(4): 119-122.
[13] LIU Shu-bo, LI Yan-min and LIU Meng-jun. Privacy-preserving for Location-based Service over Encrypted Data Search [J]. Computer Science, 2015, 42(4): 101-105.
[14] SHI Lu-sheng and QIN Xiao-lin. Privacy-preserving Data Aggregation Algorithm with Integrity Verification [J]. Computer Science, 2013, 40(11): 197-202.
[15] . Research on Anonymity Technique for Personalization Privacy-preserving Data Publishing [J]. Computer Science, 2012, 39(4): 168-171.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.