Computer Science

Computer Science ›› 2023, Vol. 50 ›› Issue (6A): 230200188-9.doi: 10.11896/jsjkx.230200188

• Information Security • Previous Articles    

Robust Federated Learning Algorithm Based on Adaptive Weighting

ZHANG Lianfu1,2, TAN Zuowen1   

  1. 1 Department of Computer Science, Technology, School of Information Management, Jiangxi University of Finance, Economics, Nanchang 330013, China;
    2 College of Mathematics and Computer Science,Yichun University,Yichun,Jiangxi 336000,China
  • Online:2023-06-10 Published:2023-06-12
  • About author:ZHANG Lianfu,born in 1978,Ph.D candidate,is a member of China Computer Federation.His main research interests include information security and privacy-preserving machine learning. TAN Zuowen,born in 1967,Ph.D,professor,PhD supervisor,is a member of China Computer Federation.His main research interests include cryptography,blockchain and privacy-preserving machine learning.
  • Supported by:
    National Natural Science Foundation of China(61862028) and Youth Projects Science and Technology of Jiangxi Provincial Department of Education(GJJ210529).

PDF (PC)

708

Abstract: Federated learning allows multiple data owners to jointly train machine learning models without sharing private training data.However,studies have shown that FL is vulnerable to Byzantine attacks and privacy breaches,this problem has not been well addressed by existing studies.In the federated learning scenario,protecting FL from Byzantine attacks while considering performance,efficiency,privacy,number of attackers,simplicity and feasibility is a challenging problem.To solve this problem,a privacy preserving robust federal learning algorithm DP-FedAWA is proposed based on l2-norm distance and quadratic normalization.The proposed algorithm does not require any assumptions outside the training process and can deal with a few or a lot of attackers adaptively.In no defense setting,DP-FedAvg is used as the comparison baseline,while Krum and Median are used as the comparison baseline in the defense setting.Extensive experiments on MedMNIST2D data set confirm that the proposed DP-FedAWA algorithm is safe and robust to malicious clients,and comprehensively outperforms the existing Krum and Median in Accuracy,Precision,Recall and F1-Score.

Key words: Adaptive weighting, l2-norm distance, Quadratic normalization, Byzantine attacks, Robust federated learning, Differential privacy

CLC Number: 

  • TP391
[1]MCKINNEY S M,SIENIEK M,GODBOLE V,et al.International evaluation of an AI system for breast cancer screening[J].Nature,2020,577(7788):89-94.
[2]LEE J,SUN J,WANG F,et al.Privacy-preserving patient similarity learning in a federated environment:development and analysis[J].JMIR Nedical Informatics,2018,6(2):e7744.
[3]ELSHAFEEY N,KOTROTSOU A,HASSAN A,et al.Multicenter study demonstrates radiomic features derived from magnetic resonance perfusion images identify pseudoprogression in glioblastoma[J].Nature Communications,2019,10(1):3170.
[4]KAISSIS G,ZIEGELMAYER S,LOHÖFER F,et al.A machine learning model for the prediction of survival and tumor subtype in pancreatic ductal adenocarcinoma from preoperative diffusion-weighted imaging[J].European Radiology Experimental,2019,3(1):1-9.
[5]LU H,ARSHAD M,THORNTON A,et al.A mathematical-descriptor of tumor-mesoscopic-structure from computed-tomography images annotates prognostic-and molecular-phenotypes of epithelial ovarian cancer[J].Nature Communications,2019,10(1):764.
[6]FREDRIKSON M,JHA S,RISTENPART T.Model inversionattacks that exploit confidence information and basic counte-rmeasures[C]//Proceedings of the 22nd ACM SIGSAC Confe-rence on Computer and Communications Security.2015:1322-1333.
[7]BLANCHARD P,EL MHAMDI E M,GUERRAOUI R,et al.Machine learning with adversaries:Byzantine tolerant gradient descent[J].Advances in Neural Information Processing Systems,2017,30.
[8]ABADI M,CHU A,GOODFELLOW I,et al.Deep learning with differential privacy[C]//Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security.2016:308-318.
[9]DWORK C.Differential privacy[C]//Proceedings of the Automata,Languages and Programming:33rd International Colloquium(ICALP 2006).Venice,Italy,2006:1-12.
[10]GEYER R C,KLEIN T,NABI M.Differentially private federated learning:A client level perspective[J].arXiv:171207557,2017.
[11]SATHYA S S,VEPAKOMMA P,RASKAR R,et al.A review of homomorphic encryption libraries for secure computation[J].arXiv:181202428,2018.
[12]XU G,LI H,ZHANG Y,et al.Privacy-preserving federateddeep learning with irregular users[J].IEEE Transactions on Dependable and Secure Computing,2020,19(2):1364-1381.
[13]KELLER M,PASTRO V,ROTARU D.Overdrive:making SPDZ great again[C]//37th Annual International Conference on the Theory and Applications of Cryptographic Techniques(EUROCRYPT 2018).Tel Aviv,Israel,2018:158-189.
[14]BOYLE E,GILBOA N,ISHAI Y.Function secret sharing:Improvements and extensions[C]//Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security.2016:1292-1303.
[15]MOHASSEL P,ZHANG Y.Secureml:A system for scalableprivacy-preserving machine learning[C]//Proceedings of the 2017 IEEE Symposium on Security and Privacy(SP).IEEE,2017:19-38.
[16]TANG L T,WANG D,ZHANG L F,et al.Federated learning scheme based on secure multi-party computation and differential privacy[J].Computer Science,2022,49(9):297-305.
[17]LI S,CHENG Y,LIU Y,et al.Abnormal client behavior detection in federated learning[J].arXiv:191009933,2019.
[18]XIE C,KOYEJO S,GUPTA I.Zeno:Distributed stochastic gradient descent with suspicion-based fault-tolerance[C]//Procee-dings of the International Conference on Machine Learning.PMLR,2019:6893-6901.
[19]CAO X,LAI L.Distributed gradient descent algorithm robust to an arbitrary number of byzantine attackers[J].IEEE Transactions on Signal Processing,2019,67(22):5850-5864.
[20]CAO X,FANG M,LIU J,et al.Fltrust:Byzantine-robust federated learning via trust bootstrapping[J].arXiv:201213995,2022.
[21]BLANCHARD P,MHAMDI E,GUERRAOUI R,et al.Ma-chine learning with adversaries:byzantine tolerant gradient descent[C]//Proceedings of the Neural Information Processing Systems.2017.
[22]XIA Q,TAO Z,HAO Z,et al.FABA:an algorithm for fast aggregation against byzantine attacks in distributed neural networks[C]//Proceedings of the IJCAI.2019.
[23]YIN D,CHEN Y,KANNAN R,et al.Byzantine-robust distributed learning:Towards optimal statistical rates[C]//Proceedings of the International Conference on Machine Learning.PMLR,2018:5650-5659.
[24]GUERRAOUI R,ROUAULT S.The hidden vulnerability ofdistributed learning in byzantium[C]//Proceedings.of the International Conference on Machine Learning.PMLR,2018:3521-3530.
[25]CHEN Y,SU L,XU J.Distributed statistical machine learning in adversarial settings:Byzantine gradient descent[C]//Proceedings of the ACM on Measurement and Analysis of Computing Systems.2017:1-25.
[26]PILLUTLA K,KAKADE S M,HARCHAOUI Z.Robust aggregation for federated learning[J].IEEE Transactions on Signal Processing,2022,70:1142-1154.
[27]YAN M,LIN Y,NIE Z S,et al.Training Method to Improve Robustness of Federated Learning[J].Computer Science,2022,49(S1):496-501.
[28]HONGYAN C,VIRAT S,REZA S,et al.Cronus:Robust and heterogeneous collaborative learning with black-box knowledge transfer[J].arXiv:191211279,2019.
[29]MIAO Y,LIU Z,LI H,et al.Privacy-preserving Byzantine-robust federated learning via blockchain systems[J].IEEE Transactions on Information Forensics and Security,2022,17:2848-2861.
[30]TANG X,SHEN M,LI Q,et al.PILE:Robust Privacy-Preserving Federated Learning via Verifiable Perturbations[J].IEEE Transactions on Dependable and Secure Computing,2023:1-18.
[31]TAN Z,ZHANG L.Survey on privacy preserving techniques for machine learning[J].J Softw,2020,31(7):2127-2156.
[32]DWORK C,ROTH A.The algorithmic foundations of differential privacy[J].Foundations and Trends© in Theoretical Computer Science,2014,9(3):211-407.
[33]MCMAHAN H B,RAMAGE D,TALWAR K,et al.Learningdifferentially private recurrent language models[J].arXiv:171006963,2017.
[34]HITAJ B,ATENIESE G,PEREZ-CRUZ F.Deep models under the GAN:information leakage from collaborative deep learning[C]//Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security.2017:603-618.
[35]SHOKRI R,STRONATI M,SONG C,et al.Membership infe-rence attacks against machine learning models[C]//Proceedings of the 2017 IEEE Symposium on Security and Privacy(SP).IEEE,2017:3-18.
[36]YANG J,SHI R,WEI D,et al.Medmnist v2:A large-scale lightweight benchmark for 2d and 3d biomedical image classification[J].arXiv:211014795,2021.
[1] LI Kejia, HU Xuexian, CHEN Yue, YANG Hongjian, XU Yang, LIU Yang. Differential Privacy Linear Regression Algorithm Based on Principal Component Analysis andFunctional Mechanism [J]. Computer Science, 2023, 50(8): 342-351.
[2] LI Rongchang, ZHENG Haibin, ZHAO Wenhong, CHEN Jinyin. Data Reconstruction Attack for Vertical Graph Federated Learning [J]. Computer Science, 2023, 50(7): 332-338.
[3] ZHAO Yuqi, YANG Min. Review of Differential Privacy Research [J]. Computer Science, 2023, 50(4): 265-276.
[4] LIU Likang, ZHOU Chunlai. RCP:Mean Value Protection Technology Under Local Differential Privacy [J]. Computer Science, 2023, 50(2): 333-345.
[5] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[6] HUANG Jue, ZHOU Chun-lai. Frequency Feature Extraction Based on Localized Differential Privacy [J]. Computer Science, 2022, 49(7): 350-356.
[7] WANG Mei-shan, YAO Lan, GAO Fu-xiang, XU Jun-can. Study on Differential Privacy Protection for Medical Set-Valued Data [J]. Computer Science, 2022, 49(4): 362-368.
[8] KONG Yu-ting, TAN Fu-xiang, ZHAO Xin, ZHANG Zheng-hang, BAI Lu, QIAN Yu-rong. Review of K-means Algorithm Optimization Based on Differential Privacy [J]. Computer Science, 2022, 49(2): 162-173.
[9] QU Xiang-mou, WU Ying-bo, JIANG Xiao-ling. Federated Data Augmentation Algorithm for Non-independent and Identical Distributed Data [J]. Computer Science, 2022, 49(12): 33-39.
[10] SHI Kun, ZHOU Yong, ZHANG Qi-liang, JIANG Shun-rong. Privacy-preserving Scheme of Energy Trading Data Based on Consortium Blockchain [J]. Computer Science, 2022, 49(11): 335-344.
[11] HE Yuan, XING Chang-you, ZHANG Guo-min, SONG Li-hua, YU Hang. Differential Privacy Based Fingerprinting Obfuscation Mechanism Towards NetworkReconnaissance Deception [J]. Computer Science, 2022, 49(11): 351-359.
[12] WANG Xiu-jun, MO Lei, ZHENG Xiao, GAO Yun-quan. Adaptive Histogram Publishing Algorithm for Sliding Window of Data Stream [J]. Computer Science, 2022, 49(10): 344-352.
[13] DONG Xiao-mei, WANG Rui, ZOU Xin-kai. Survey on Privacy Protection Solutions for Recommended Applications [J]. Computer Science, 2021, 48(9): 21-35.
[14] SUN Lin, PING Guo-lou, YE Xiao-jun. Correlation Analysis for Key-Value Data with Local Differential Privacy [J]. Computer Science, 2021, 48(8): 278-283.
[15] ZHANG Xue-jun, YANG Hao-ying, LI Zhen, HE Fu-cun, GAI Ji-yang, BAO Jun-da. Differentially Private Location Privacy-preserving Scheme withSemantic Location [J]. Computer Science, 2021, 48(8): 300-308.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.