计算机科学

计算机科学 ›› 2023, Vol. 50 ›› Issue (9): 62-67.doi: 10.11896/jsjkx.220700174

• 数据安全 • 上一篇    下一篇

抗推理攻击的隐私增强联邦学习算法

赵宇豪1, 陈思光1, 苏健2   

  1. 1 南京邮电大学物联网学院 南京 210003
    2 南京信息工程大学计算机学院 南京 210044
  • 收稿日期:2022-07-18 修回日期:2023-01-06 出版日期:2023-09-15 发布日期:2023-09-01
  • 通讯作者: 陈思光(sgchen@njupt.edu.cn)
  • 作者简介:(zyh19981202@163.com)
  • 基金资助:
    国家自然科学基金(61971235);江苏省“333高层次人才培养工程”资助;中国博士后科学基金(面上一等资助)(2018M630590);江苏省博士后科研资助计划(2021K501C);南京邮电大学“1311”人才计划

Privacy-enhanced Federated Learning Algorithm Against Inference Attack

ZHAO Yuhao1, CHEN Siguang1, SU Jian2   

  1. 1 School of Internet of Things,Nanjing University of Posts and Telecommunications,Nanjing 210003,China
    2 School of Computer Science,Nanjing University of Information Science and Technology,Nanjing 210044,China
  • Received:2022-07-18 Revised:2023-01-06 Online:2023-09-15 Published:2023-09-01
  • About author:ZHAO Yuhao,born in 1998,postgra-duate.His main research interest is fe-derated learning.
    CHEN Siguang,born in 1984,Ph.D,professor.His main research interests include edge intelligence and AIoT.
  • Supported by:
    National Natural Science Foundation of China(61971235),333 High-level Talents Training Project of Jiangsu Province,China Postdoctoral Science Foundation(2018M630590),Jiangsu Planned Projects for Postdoctoral Research Funds(2021K501C) and 1311 Talents Plan of NJUPT.

PDF (PC)

2347

摘要： 联邦学习在保证各分布式客户端训练数据不出本地的情况下,由中心服务器收集梯度协同训练全局网络模型,具有良好的性能与隐私保护优势。但研究表明,联邦学习存在梯度传递引起的数据隐私泄漏问题。针对现有安全联邦学习算法存在的模型学习效果差、计算开销大和防御攻击种类单一等问题,提出了一种抗推理攻击的隐私增强联邦学习算法。首先,构建了逆推得到的训练数据与训练数据距离最大化的优化问题,基于拟牛顿法求解该优化问题,获得具有抗推理攻击能力的新特征。其次,利用新特征生成梯度实现梯度重构,基于重构后的梯度更新网络模型参数,可提升网络模型的隐私保护能力。最后,仿真结果表明所提算法能够同时抵御两类推理攻击,并且相较于其他安全方案,所提算法在保护效果与收敛速度上更具优势。

关键词: 联邦学习, 推理攻击, 隐私保护, 梯度扰动

Abstract: In federated learning,each distributed client does not need to transmit local training data,the central server jointly trains the global model by gradient collection,it has good performance and privacy protection advantages.However,it has been demonstrated that gradient transmission may lead to the privacy leakage problem in federated learning.Aiming at the existing problems of current secure federated learning algorithms,such as poor model learning effect,high computational cost,and single attack defense,this paper proposes a privacy-enhanced federated learning algorithm against inference attack.First,an optimization problem of maximizing the distance between the training data obtained by inversion and the training data is formulated.The optimization problem is solved based on the quasi-Newton method to obtain new features with anti-inference attack ability.Second,the gradient reconstruction is achieved by using new features to generate gradients.The model parameters are updated based on the reconstructed gradients,which can improve the privacy protection capability of the model.Finally,simulation results show that the proposed algorithm can resist two types of inference attacks simultaneously,and it has significant advantages in protection effect and convergence speed compared with other secure schemes.

Key words: Federated learning, Inference attack, Privacy preservation, Gradient perturbation

中图分类号: 

  • TP393
[1]MCMAHAN B,MOORE E,RAMAGE D,et al.Communication-efficient learning of deep networks from decentralized data [C]//Proceedings of the 20th International Conference on Artificial Intelligence and Statistics(AISTATS).2016:1273-1282.
[2]YANG Q,LIU Y,CHEN T,et al.Federated machine learning:Conceptand applications [J].ACM Transactions on Intelligent Systems and Technology,2019,10(2):1-19.
[3]BONAWITZ K,EICHNER H,GRIESKAMP W,et al.Towards federated learning at scale:System design [C]//Proceedings of Machine Learning and Systems(MLSys).2019:374-388.
[4]LI T,SAHU A K,TALWALKAR A,et al.Federated learning:Challenges,methods,and future directions [J].IEEE Signal Processing Magazine,2020,37(3):50-60.
[5]ZHU L,LIU Z,HAN S.Deep leakage from gradients [C]//Pro-ceedings of Advances in Neural Information Processing Systems(NIPS).2019:17-31.
[6]GEIPING J,BAUERMEISTER H,DRÖGE H,et al.Inverting gradients-how easy is it to break privacy in federated learning? [C]//Proceedings of Advances in Neural Information Proces-sing Systems(NIPS).2020:16937-16947.
[7]WANG Z,SONG M,ZHANG Z,et al.Beyond inferring class representatives:User-level privacy leakage from federated lear-ning[C]//Proceedings of IEEE International Conference on Computer Communications(INFOCOM).2019:2512-2520.
[8]LIU J,MENG X.Survey on Privacy-Preserving Machine Lear-ning[J].Journal of Computer Research and Development,2020,57(2):346-362.
[9]WEI K,LI J,DING M,et al.Federated learning with differentialprivacy:Algorithms and performance analysis [J].IEEE Tran-sactions on Information Forensics and Security,2020,15:3454-3469.
[10]MCMAHAN H B,RAMAGE D,TALWAR K,et al.Learning differentially private recurrent language models [C]//Procee-dings of International Conference on Learning Representations(ICLR).2018:171-182.
[11]TRUEX S,LIU L,CHOW K H,et al.LDP-Fed:Federatedlearning with local differential privacy [C]//Proceedings of the Third ACM International Workshop on Edge Systems(EdgeSys).2020:61-66.
[12]BONAWITZ K,IVANOV V,KREUTER B,et al.Practical secure aggregation for privacy-preserving machine learning [C]//Proceedings of ACM SIGSAC Conference on Computer and Communications Security(CCS).2017:1175-1191.
[13]LIU Y,KANG Y,XING C,et al.A secure federated transfer learning framework[J].IEEE Intelligent Systems,2020,35(4):70-82.
[14]WEI W,LIU L,WUT Y,et al.Gradient-leakage resilient federa-ted learning [C]//Proceedings of the 41st IEEE International Conference on Distributed Computing Systems(ICDCS).2021:797-807.
[15]WU N,FAROKHI F,SMITH D,et al.The value of collaboration in convex machine learning with differential privacy [C]//Proceedings of IEEE Symposium on Security and Privacy(SP).2020:304-317.
[16]LIN Y,HAN S,MAO H,et al.Deep gradient compression:Reducing the communication bandwidth for distributed training[C]//Proceedings of International Conference on Learning Representations(ICLR).2017:1-12.
[17]MARTINS P,SOUSA L,MARIANO A.A survey on fully homomorphic encryption:An engineering perspective [J].ACM Computing Surveys,2017,50(6):1-33.
[18]ACAR A,AKSU H,ULUAGAC A S,et al.A survey on homomorphic encryption schemes:Theory and implementation [J].ACM Computing Surveys,2018,51(4):1-35.
[19]ZHANG Z,FU Y,HE N,GAO T.Research on Federated Deep Neural Network Model for Data Privacy Preserving[J].Acta Automatica Sinica,2022,48(5):1273-1284.
[20]SUN J,LI A,WANG B,et al.Soteria:Provable defense against privacy leakage in federated learning from representation perspective [C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).2021:9311-9319.
[21]JIANG B,LI J,WANG H,et al.Privacy-Preserving federatedlearning for industrial edge computing via hybrid differential privacy and adaptive compression [J].IEEE Transactions on Industrial Informatics,2023,19(2):1136-1144.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发