计算机科学

计算机科学 ›› 2023, Vol. 50 ›› Issue (6A): 230200188-9.doi: 10.11896/jsjkx.230200188

• 信息安全 • 上一篇    

一种基于自适应加权的鲁棒联邦学习算法

张连福1,2, 谭作文1   

  1. 1 江西财经大学信息管理学院计算机科学与技术系 南昌 330013;
    2 宜春学院数学与计算机科学学院 江西 宜春 336000
  • 出版日期:2023-06-10 发布日期:2023-06-12
  • 通讯作者: 谭作文(tanzyw@163.com)
  • 作者简介:(zlf_jx@163.com)
  • 基金资助:
    国家自然科学基金(61862028);江西省教育厅青年科技项目(GJJ210529)

Robust Federated Learning Algorithm Based on Adaptive Weighting

ZHANG Lianfu1,2, TAN Zuowen1   

  1. 1 Department of Computer Science, Technology, School of Information Management, Jiangxi University of Finance, Economics, Nanchang 330013, China;
    2 College of Mathematics and Computer Science,Yichun University,Yichun,Jiangxi 336000,China
  • Online:2023-06-10 Published:2023-06-12
  • About author:ZHANG Lianfu,born in 1978,Ph.D candidate,is a member of China Computer Federation.His main research interests include information security and privacy-preserving machine learning. TAN Zuowen,born in 1967,Ph.D,professor,PhD supervisor,is a member of China Computer Federation.His main research interests include cryptography,blockchain and privacy-preserving machine learning.
  • Supported by:
    National Natural Science Foundation of China(61862028) and Youth Projects Science and Technology of Jiangxi Provincial Department of Education(GJJ210529).

PDF (PC)

720

摘要： 联邦学习(Federated Learning,FL)允许多个数据所有者联合训练机器学习模型,而无需他们共享私有训练数据。然而,研究表明,FL容易同时遭受拜占庭攻击和隐私泄露威胁,现有的研究都没有很好地解决这一问题。在联邦学习场景中,保护FL免受拜占庭攻击,同时考虑性能、效率、隐私、攻击者数量、简单可行等问题,是一个极具挑战性的问题。为解决这一问题,基于l2范数和两次归一化方法提出了一种隐私保护鲁棒联邦学习算法DP-FedAWA。提出的算法不需要训练过程之外的任何假设,并且可以自适应地处理少量和大量的攻击者。无防御设置下选用DP-FedAvg作为比较基线,防御设置下选用Krum和Median作为比较基线。MedMNIST2D数据集上的广泛实验证实了,DP-FedAWA算法是安全的,对恶意客户端具有很好的鲁棒性,在Accuracy,Precision,Recall和F1 -Score等性能指标上全面优于现有的Krum和Median算法。

关键词: 自适应加权, l2范数距离, 两次归一化, 拜占庭攻击, 鲁棒联邦学习, 差分隐私

Abstract: Federated learning allows multiple data owners to jointly train machine learning models without sharing private training data.However,studies have shown that FL is vulnerable to Byzantine attacks and privacy breaches,this problem has not been well addressed by existing studies.In the federated learning scenario,protecting FL from Byzantine attacks while considering performance,efficiency,privacy,number of attackers,simplicity and feasibility is a challenging problem.To solve this problem,a privacy preserving robust federal learning algorithm DP-FedAWA is proposed based on l2-norm distance and quadratic normalization.The proposed algorithm does not require any assumptions outside the training process and can deal with a few or a lot of attackers adaptively.In no defense setting,DP-FedAvg is used as the comparison baseline,while Krum and Median are used as the comparison baseline in the defense setting.Extensive experiments on MedMNIST2D data set confirm that the proposed DP-FedAWA algorithm is safe and robust to malicious clients,and comprehensively outperforms the existing Krum and Median in Accuracy,Precision,Recall and F1-Score.

Key words: Adaptive weighting, l2-norm distance, Quadratic normalization, Byzantine attacks, Robust federated learning, Differential privacy

中图分类号: 

  • TP391
[1]MCKINNEY S M,SIENIEK M,GODBOLE V,et al.International evaluation of an AI system for breast cancer screening[J].Nature,2020,577(7788):89-94.
[2]LEE J,SUN J,WANG F,et al.Privacy-preserving patient similarity learning in a federated environment:development and analysis[J].JMIR Nedical Informatics,2018,6(2):e7744.
[3]ELSHAFEEY N,KOTROTSOU A,HASSAN A,et al.Multicenter study demonstrates radiomic features derived from magnetic resonance perfusion images identify pseudoprogression in glioblastoma[J].Nature Communications,2019,10(1):3170.
[4]KAISSIS G,ZIEGELMAYER S,LOHÖFER F,et al.A machine learning model for the prediction of survival and tumor subtype in pancreatic ductal adenocarcinoma from preoperative diffusion-weighted imaging[J].European Radiology Experimental,2019,3(1):1-9.
[5]LU H,ARSHAD M,THORNTON A,et al.A mathematical-descriptor of tumor-mesoscopic-structure from computed-tomography images annotates prognostic-and molecular-phenotypes of epithelial ovarian cancer[J].Nature Communications,2019,10(1):764.
[6]FREDRIKSON M,JHA S,RISTENPART T.Model inversionattacks that exploit confidence information and basic counte-rmeasures[C]//Proceedings of the 22nd ACM SIGSAC Confe-rence on Computer and Communications Security.2015:1322-1333.
[7]BLANCHARD P,EL MHAMDI E M,GUERRAOUI R,et al.Machine learning with adversaries:Byzantine tolerant gradient descent[J].Advances in Neural Information Processing Systems,2017,30.
[8]ABADI M,CHU A,GOODFELLOW I,et al.Deep learning with differential privacy[C]//Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security.2016:308-318.
[9]DWORK C.Differential privacy[C]//Proceedings of the Automata,Languages and Programming:33rd International Colloquium(ICALP 2006).Venice,Italy,2006:1-12.
[10]GEYER R C,KLEIN T,NABI M.Differentially private federated learning:A client level perspective[J].arXiv:171207557,2017.
[11]SATHYA S S,VEPAKOMMA P,RASKAR R,et al.A review of homomorphic encryption libraries for secure computation[J].arXiv:181202428,2018.
[12]XU G,LI H,ZHANG Y,et al.Privacy-preserving federateddeep learning with irregular users[J].IEEE Transactions on Dependable and Secure Computing,2020,19(2):1364-1381.
[13]KELLER M,PASTRO V,ROTARU D.Overdrive:making SPDZ great again[C]//37th Annual International Conference on the Theory and Applications of Cryptographic Techniques(EUROCRYPT 2018).Tel Aviv,Israel,2018:158-189.
[14]BOYLE E,GILBOA N,ISHAI Y.Function secret sharing:Improvements and extensions[C]//Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security.2016:1292-1303.
[15]MOHASSEL P,ZHANG Y.Secureml:A system for scalableprivacy-preserving machine learning[C]//Proceedings of the 2017 IEEE Symposium on Security and Privacy(SP).IEEE,2017:19-38.
[16]TANG L T,WANG D,ZHANG L F,et al.Federated learning scheme based on secure multi-party computation and differential privacy[J].Computer Science,2022,49(9):297-305.
[17]LI S,CHENG Y,LIU Y,et al.Abnormal client behavior detection in federated learning[J].arXiv:191009933,2019.
[18]XIE C,KOYEJO S,GUPTA I.Zeno:Distributed stochastic gradient descent with suspicion-based fault-tolerance[C]//Procee-dings of the International Conference on Machine Learning.PMLR,2019:6893-6901.
[19]CAO X,LAI L.Distributed gradient descent algorithm robust to an arbitrary number of byzantine attackers[J].IEEE Transactions on Signal Processing,2019,67(22):5850-5864.
[20]CAO X,FANG M,LIU J,et al.Fltrust:Byzantine-robust federated learning via trust bootstrapping[J].arXiv:201213995,2022.
[21]BLANCHARD P,MHAMDI E,GUERRAOUI R,et al.Ma-chine learning with adversaries:byzantine tolerant gradient descent[C]//Proceedings of the Neural Information Processing Systems.2017.
[22]XIA Q,TAO Z,HAO Z,et al.FABA:an algorithm for fast aggregation against byzantine attacks in distributed neural networks[C]//Proceedings of the IJCAI.2019.
[23]YIN D,CHEN Y,KANNAN R,et al.Byzantine-robust distributed learning:Towards optimal statistical rates[C]//Proceedings of the International Conference on Machine Learning.PMLR,2018:5650-5659.
[24]GUERRAOUI R,ROUAULT S.The hidden vulnerability ofdistributed learning in byzantium[C]//Proceedings.of the International Conference on Machine Learning.PMLR,2018:3521-3530.
[25]CHEN Y,SU L,XU J.Distributed statistical machine learning in adversarial settings:Byzantine gradient descent[C]//Proceedings of the ACM on Measurement and Analysis of Computing Systems.2017:1-25.
[26]PILLUTLA K,KAKADE S M,HARCHAOUI Z.Robust aggregation for federated learning[J].IEEE Transactions on Signal Processing,2022,70:1142-1154.
[27]YAN M,LIN Y,NIE Z S,et al.Training Method to Improve Robustness of Federated Learning[J].Computer Science,2022,49(S1):496-501.
[28]HONGYAN C,VIRAT S,REZA S,et al.Cronus:Robust and heterogeneous collaborative learning with black-box knowledge transfer[J].arXiv:191211279,2019.
[29]MIAO Y,LIU Z,LI H,et al.Privacy-preserving Byzantine-robust federated learning via blockchain systems[J].IEEE Transactions on Information Forensics and Security,2022,17:2848-2861.
[30]TANG X,SHEN M,LI Q,et al.PILE:Robust Privacy-Preserving Federated Learning via Verifiable Perturbations[J].IEEE Transactions on Dependable and Secure Computing,2023:1-18.
[31]TAN Z,ZHANG L.Survey on privacy preserving techniques for machine learning[J].J Softw,2020,31(7):2127-2156.
[32]DWORK C,ROTH A.The algorithmic foundations of differential privacy[J].Foundations and Trends© in Theoretical Computer Science,2014,9(3):211-407.
[33]MCMAHAN H B,RAMAGE D,TALWAR K,et al.Learningdifferentially private recurrent language models[J].arXiv:171006963,2017.
[34]HITAJ B,ATENIESE G,PEREZ-CRUZ F.Deep models under the GAN:information leakage from collaborative deep learning[C]//Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security.2017:603-618.
[35]SHOKRI R,STRONATI M,SONG C,et al.Membership infe-rence attacks against machine learning models[C]//Proceedings of the 2017 IEEE Symposium on Security and Privacy(SP).IEEE,2017:3-18.
[36]YANG J,SHI R,WEI D,et al.Medmnist v2:A large-scale lightweight benchmark for 2d and 3d biomedical image classification[J].arXiv:211014795,2021.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发