计算机科学

计算机科学 ›› 2023, Vol. 50 ›› Issue (5): 372-381.doi: 10.11896/jsjkx.220300239

• 信息安全 • 上一篇    下一篇

基于同态加密的神经网络模型训练方法

赵敏1,2,3, 田有亮1,2,3, 熊金波1,2,4, 毕仁万4, 谢洪涛5   

  1. 1 贵州大学公共大数据国家重点实验室 贵阳 550025
    2 贵州大学计算机科学与技术学院 贵阳 550025
    3 贵州大学密码学与数据安全研究所 贵阳 550025
    4 福建师范大学计算机与网络空间安全学院 福州 350117
    5 中国科学技术大学信息科学与技术学院 合肥 230000
  • 收稿日期:2022-03-25 修回日期:2022-12-30 出版日期:2023-05-15 发布日期:2023-05-06
  • 通讯作者: 熊金波(jbxiong@fjnu.edu.cn)
  • 作者简介:(gs.zhaom20@gzu.edu.cn)
  • 基金资助:
    国家重点研发计划(2021YFB3101100);国家自然科学基金(62272123,62272102);贵州省高层次创新型人才项目(黔科合平台人才[2020]6008);贵阳市科技计划项目(筑科合[2021]1-5,筑科合[2022]2-4);贵州省科技计划项目(黔科合平台人才[2020]5017,黔科合支撑[2022]一般065)

Neural Network Model Training Method Based on Homomorphic Encryption

ZHAO Min1,2,3, TIAN Youliang1,2,3, XIONG Jinbo1,2,4, BI Renwan4, XIE Hongtao5   

  1. 1 State Key Laboratory of Public Big Date,Guizhou University,Guiyang 550025,China
    2 College of Computer Science and Technology,Guizhou University,Guiyang,550025,China
    3 Institute of Cryptography & Data Security,Guizhou University,Guiyang 550025,China
    4 College of Computer and Cyber Security,Fujian Normal University,Fuzhou 350117,China
    5 School of Information Science and Technology,University of Science and Technology of China,Hefei 230000,China
  • Received:2022-03-25 Revised:2022-12-30 Online:2023-05-15 Published:2023-05-06
  • About author:ZHAO Min,born in 1995,postgraduate.Her main research interests include secure machine learning and privacy protection.
    XIONG Jinbo,born in 1981,Ph.D,professor,Ph.D supervisor.His main research interests include secure deep learning,mobile crowdsensing security and privacy protection.
  • Supported by:
    National Key Research and Development Program of China(2021YFB3101100),National Natural Science Foundation of China(62272123,62272102),Project of High-level Innovative Talents of Guizhou Province([2020]6008),Science and Technology Program of Guiyang([2021]1-5,[2022]2-4) and Science and Technology Program of Guizhou Province([2020]5017,[2022]065).

PDF (PC)

800

摘要： 针对云环境下数据隐私泄露与基于同态加密的隐私保护神经网络中精度不足的问题,文中提出了一种双服务器协作的隐私保护神经网络训练(PPNT)方案,在云服务器协同训练过程中实现了对数据传输、计算过程及模型参数的隐私保护。首先,为避免使用多项式近似方法实现指数和比较等非线性函数,并提高非线性函数的计算精度,基于Paillier半同态加密方案和加法秘密共享技术设计了一系列基础安全计算协议;其次,在已设计的安全计算协议基础上,构造了神经网络中的全连接层、激活层、Softmax层及反向传播相应的安全计算协议,以实现PPNT方案;最后,通过理论与安全性分析,证明了PPNT方案的正确性及安全性。性能实验结果显示,与PPMLaaS方案相比,PPNT方案的模型精度提高了1.7%,且在安全计算过程中支持客户端离线。

关键词: Paillier半同态加密, 加法秘密共享, 安全计算协议, 隐私保护, 模型训练

Abstract: Aiming at the problem of data privacy leakage in cloud environment and insufficient accuracy in the privacy-preserving neural network based on homomorphic encryption,a privacy-preserving neural network training scheme(PPNT) is proposed for collaborative dual cloud servers,to achieve the goal of data transmission,computing security and model parameter under the collaborative training process of dual cloud servers.Firstly,in order to avoid using polynomial approximation method to realize nonlinear functions such as exponent and comparison,and improve the calculation accuracy of nonlinear function,a series of secure computing protocols are designed based on Paillier partially homomorphic encryption technology and additive secret sharing scheme.Furthermore,corresponding secure computing protocols of full connection layer,activation layer,softmax layer and back propagation in neural network are constructed to realize PPNT based on the designed secure computing protocols.Finally,theoretical and security analysis guarantees the correctness and security of PPNT.The actual performance results show that compared with the dual server scheme--privacy protection machine learning as a service(PPMLaaS),the model accuracy of PPNT improves by 1.7%,and supports the client offline in the process of secure computing.

Key words: Paillier partially homomorphic encryption, Additive secret sharing, Secure computing protocol, Privacy-preserving, Model training

中图分类号: 

  • TP309.2
[1]MA Z,LIU Y,LIU X,et al.Lightweight privacy-preserving ensemble classification for face recognition[J].IEEE Internet of Things Journal,2019,6(3):5778-5790.
[2]LUO X,LI L,WAN H,et al.Phone keypad voice recognition:an integrated experiment for digital signal processing education[C]//Proceedings of the 2020 IEEE Frontiers in Education Conference.Piscataway:IEEE Press,2020:1-4.
[3]LI Z Y,GUI X L,GU Y J,et al.Survey on homomorphic encryption algorithm and its application in the privacy-preserving for cloud computing[J].Journal of Software,2018,29(7):1830-1851.
[4]TAN Z W,ZHANG L F.Survey on privacy preserving techniques for machine learning[J].Journal of Software,2020,31(7):2127-2156.
[5]GILAD-BACHRACH R,DOWLIN N,LAINE K,et al.Cryp-tonets:applying neural networks to encrypted data with high throughput and accuracy[C]//International Conference on Machine Learning.New York:ACM Press,2016:201-210.
[6]HESAMIFARD E,TAKABI H,GHASEMI M.Cryptodl:Deep neural networks over encrypted data[J].arXiv:1711.05189,2017.
[7]CHOU E,BEAL J,LEVY D,et al.Faster cryptonets:leveraging sparsity for real-world encrypted inference[J].arXiv:1811.09953,2018.
[8]CHABANNE H,DE W A,MILGRAM J,et al.Privacy-preserving classification on deep neural network[J／OL].Cryptology ePrint Archive,2017,1-35.http://eprint.iacr.org/2017/035.
[9]JUVEKAR C,VALKUNTANATHAN V,CHANDRAKASAN A.{GAZELLE}:A low latency framework for secure neural network inference[C]//27th USENIX Security Symposium({USENIX} Security 18).Berkeley:USENIX Association,2018:1651-1669.
[10]BADAWI A,CHAO J,JIE L,et al.Towards the alexnet mo-ment for homomorphic encryption:hcnn,the first homomorphic cnn on encrypted data with gpus[J].IEEE Transactions on Emerging Topics in Computing,2021,9(3):1330-1343.
[11]HAN K,HONG S,CHEON J H,et al.Logistic regression onhomomorphic encrypted data at scale[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Menlo Park:AAAI Press,2019:9466-9471.
[12]BRAKERSKI Z,GENTRY C,VAIKUNTANATHAN V.(Le-veled) fully homomorphic encryption without bootstrapping[J].ACM Transactions on Computation Theory(TOCT),2014,6(3):1-36.
[13]ZHANG Q,WANG C,WU H,et al.GELU-Net:a globally encrypted,locally unencrypted deep neural network for privacy-preserved learning[C]//Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence.Stockholm:IJCAI.2018:3933-3939.
[14]BOURSE F,MINELLI M,MINIHOLD M,et al.Fast homomorphic evaluation of deep discretized neural networks[C]//Annual International Cryptology Conference.Berlin:Springer,2018:483-512.
[15]CHILLOTTI I,GAMA N,GEORGIEVA M,et al.Faster fully homomorphic encryption:Bootstrapping in less than 0.1 seconds[C]//International Conference on the Theory and Application of Cryptology and Information Security.Berlin:Springer,2016:3-33.
[16]HESAMIFARD E,TAKABI H,GHASEMI M,et al.Privacy-preserving machine learning in cloud[C]//Proceedings of the 2017 on Cloud Computing Security Workshop.New York:ACM Press,2017:39-43.
[17]LOU Q,FENG B,CHARLES F G,et al.Glyph:fast and accurately training deep neural networks on encrypted data[J/OL].Advances in Neural Information Processing Systems,2020,33:9193-9202.https://proceedings.neurips.cc/paper/2020/hash/685ac8cadc1be5ac98da9556bc1c8d9e-Abstract.html.
[18]PAILLIER P.Public-key cryptosystems based on composite degree residuosity classes[C]//Proceedings of the International Conference on the Theory and Dpplications of Cryptographic Techniques.Berlin:Springer,1999:223-238.
[19]SHAMIR A.How to share a secret[J].Communications of the ACM,1979,22(11):612-613.
[20]LIU Y,MA Z,LIU X,et al.Privacy-preserving object detection for medical images with faster R-CNN[J／OL].IEEE Transactions on Information Forensics and Security,2022,17:69-84.https://doi.org/10.1109/TIFS.2019.2946476.
[21]XIONG J B,BI R W,TIAN Y L,et al.Towards lightweight,privacy-preserving cooperative object classification for connected autonomous vehicles[J].IEEE Internet of Things Journal,2021,9(4):2787-2801.
[22]HUANG K,LIU X,FU S,et al.A lightweight privacy-preserving CNN feature extraction framework for mobile sensing[J].IEEE Transactions on Dependable and Secure Computing,2019,18(3):1441-1455.
[23]XIONG J B,ZHOU Y J,BI R W,et al.Towards edge-collaborative,lightweight and privacy-preserving classification framework[J].Journal on Communications,2022,43(1):127-137.
[24]MA Z,LIU Y,LIU X,et al.Privacy-preserving outsourcedspeech recognition for smart IoT devices[J].IEEE Internet of Things Journal,2019,6(5):8406-8420.
[25]BI R W,CHEN Q X,XIONG J B,et al.Design method of secure computing protocol for deep neural network[J].Chinese Journal of Network and Information Security,2020,6(4):130-139.
[26]WAGH S,TOPLE S,BENHAMOUDA F,et al.Falcon:honest-majority maliciously secure framework for private deep learning[J].Privacy Enhancing Technologies,2021,2021(1):188-208.
[27]BOGDANOV D,NIITSOO M,TOFT T,et al.High-perfor-mance secure multi-party computation for data mining applications[J].International Journal of Information Security,2012,11(6):403-418.
[28]XIONG J,BI R,ZHAO M,et al.Edge-assisted privacy-preserving raw data sharing framework for connected autonomous vehicles[J].IEEE Wireless Communications,2020,27(3):24-30.
[29]XIONG J B,BI R W,CHEN Q X,et al.Towards edge-collaborative,lightweight and secure region proposal network[J].Journal on Communications,2020,41(10):188-201.
[30]HE K,ZHANG X,REN S,et al.Delving deep into rectifiers:surpassing human-level performance on imagenet classification[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society,2015:1026-1034.
[31]MOHASSEL P,ZHANG Y.Secureml:a system for scalable privacy-preserving machine learning[C]//2017 IEEE Symposium on Security and Privacy(SP).Piscataway:IEEE Press,2017:19-38.
[32]HESAMIFARD E,TAKABI H,GHASEMI M,et al.Privacy-preserving machine learning as a service[J].Proceedings on Privacy Enhancing Technologies,2018,2018(3):123-142.
[33]LIU J,JUUTI M,LU Y,et al.Oblivious neural network predictions via minionn transformations[C]//Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security.New York:ACM Press,2017:619-631.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发