基于多特征检测与自适应权重调整的鲁棒联邦学习算法

doi:10.11896/jsjkx.231100072

摘要/Abstract

摘要： 联邦学习作为一种保护隐私的分布式机器学习范式,允许多个客户端在不泄露原始训练数据的情况下协同训练全局模型。然而,由于无法直接访问客户端本地训练数据和无法监控本地训练过程,联邦学习面临各种拜占庭攻击的威胁,如数据中毒和模型篡改攻击。这些攻击旨在扰乱联邦学习模型训练过程,降低模型性能。针对此问题,尽管已有许多研究提出了不同的聚合算法,但这些方法主要聚焦于单一拜占庭攻击场景,而忽略了实际环境中可能出现的混合拜占庭攻击所带来的威胁。为应对这一难题,受净水器的原理启发,提出了一种基于多特征检测与自适应权重调整的新型拜占庭鲁棒聚合算法FL-Sieve,旨在通过多层次的筛查过滤恶意客户端。首先,算法通过角幅相似度和模型边界测度评估客户端间的特征相似性,生成相似度矩阵并计算相似性分数;接着,利用聚类算法将相似的节点归入同一簇,以确保相似的节点能够被正确分类;随后,根据预定义规则筛选潜在良性客户端;最后,根据每个客户端的信任度智能地分配权重,进一步增强防御效果和系统鲁棒性。为了验证FL-Sieve的性能,实验利用了MNIST,Fashion-MNIST和CIFAR-10这3种数据集,考虑了Non-IID数据分布情景和混合拜占庭攻击场景。混合拜占庭客户端的数量从20%递增到49%,以模拟大规模混合拜占庭客户端攻击的场景。同时也对FL-Sieve在IID和Non-IID数据分布以及单攻击场景下的性能进行了测试。实验结果表明,FL-Sieve能够有效抵御不同场景下的拜占庭攻击,即使在高达49%的混合拜占庭客户端攻击下,FL-Sieve依然能够维持较高的主任务准确率。相比之下,几种现有的经典算法存在不同程度的失效,凸显出FL-Sieve的优势。

关键词: 联邦学习, 混合拜占庭攻击, 多特征检测, 动态分配权重, 鲁棒聚合算法

Abstract: The federated learning paradigm is designed to preserve privacy by enabling multiple clients to collaboratively train a global model without compromising the original training data.However,due to the lack of direct access to local training data and monitoring capabilities during the training process,federated learning is vulnerable to various Byzantine attacks,including data poisoning and model tampering attacks.These malicious activities aim at disrupting the federated learning model training process and degrading its performance.While several studies have proposed various aggregation algorithms to address this issue,they predominantly concentrate on single Byzantine attack scenarios,often overlooking the threats associated with hybrid Byzantine attacks that can manifest in real-world environments.To address this issue,inspired by the principle of water purifiers,we propose an innovative multi-feature detection and adaptive dynamic weighting allocation algorithm called FL-Sieve for identifying Byzantine clients,aiming to filter out malicious clients through multi-level screening.Firstly,the algorithm assesses feature similarity between clients through angular range similarity and model boundary metric,generates a similarity matrix and calculates the similarity score.Then,it performs clustering to ensure that nodes with similar features are grouped together.Subsequently,it employs predefined rules to filter potential benign clients.Finally,it intelligently allocates weights based on the trustworthiness of each client,further enhancing the defense mechanisms and system robustness.To evaluate the performance of the FL-Sieve algorithm,experiments are conducted using three datasets:MNIST,Fashion-MNIST,and CIFAR-10.The experiments consider scenarios with both non-IID data distribution and hybrid Byzantine attack situations.The number of hybrid Byzantine clients increases from 20% to 49% to simulate large-scale hybrid Byzantine client attacks.Additionally,the performance of the FL-Sieve algorithm is tested in both IID and non-IID data distribution,as well as in single attack scenarios.The experimental results demonstrate that FL-Sieve effectively withstands Byzantine attacks in various scenarios,maintaining high main task accuracy even under the challenging condition of 49% hybrid Byzantine client attacks.In comparison,several existing classical algorithms exhibit varying degrees of failure,underscoring the significant advantages of the FL-Sieve algorithm.

Key words: Federated learning, Hybrid Byzantine attack, Multi-feature detection, Dynamic weight allocation, Robust aggregation algorithm

中图分类号:

TP309

王春东, 赵立扬, 张博宇, 赵永新. 基于多特征检测与自适应权重调整的鲁棒联邦学习算法[J]. 计算机科学, 2024, 51(11A): 231100072-10. https://doi.org/10.11896/jsjkx.231100072

WANG Chundong, ZHAO Liyang, ZHANG Boyu, ZHAO Yongxin. Robust Federated Learning Algorithm Based on Multi-feature Detection and Adaptive WeightAdjustment[J]. Computer Science, 2024, 51(11A): 231100072-10. https://doi.org/10.11896/jsjkx.231100072

参考文献

[1]MCMAHAN B,MOORE E,RAMAGE D,et al.Communica-tion-efficient learning of deep networks from decentralized data[C]//Artificial Intelligence and Statistics.PMLR,2017:1273-1282.
[2]LU Z,KUO-HUI Y,GERHARD H,et al.Security and Privacy for the Industrial Internet of Things:An Overview of Approaches to Safeguarding Endpoints[J].IEEE Signal Processing Magazine,2018,35(5):76-87.
[3]ZHOU C X,SUN Y,WANG D G,et al.Survey of federatedlearning research[J].Chinese Journal of Network and Information Security,2021,7(5):77-92.
[4]KHAN L U,SAAD W,HAN Z,et al.Federated Learning for Internet of Things:Recent Advances,Taxonomy,and Open Challenges[J].IEEE Communications Surveys & Tutorials,2021,23(3):1759-1799.
[5]HARD A,RAO K,MATHEWS R,et al.Federated Learning for Mobile Keyboard Prediction[J].arXiv:1181.03604,2018.
[6]LEROY D,COUCKE A,LAVRIL T,et al.Federated learning for keyword spotting[C]//IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP 2019).IEEE,2019:6341-6345.
[7]LIU Y,HUANG A,LUO Y,et al.Fedvision:An online visual object detection platform powered by federated learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:13172-13179.
[8]LI L,XU W,CHEN T,et al.RSA:Byzantine-robust stochastic aggregation methods for distributed learning from heterogeneous datasets[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2019:1544-1551.
[9]WU Z,LING Q,CHEN T,et al.Federated variance-reduced sto-chastic gradient descent with robustness to byzantine attacks[J].IEEE Transactions on Signal Processing,2020,68:4583-4596.
[10]CAO X,FANG M,LIU J,et al.Fltrust:Byzantine-robust federated learning via trust bootstrapping[C]//Network and Distributed System Security Symposium.Internet Society,2021.
[11]FANG M,CAO X,JIA J,et al,Local model poisoning attacks to byzantine-robust federated learning[C]//29^th USENIX Security Symposium(USENIX Security 20).2020:1605-1622.
[12]KAIROUZ P,MCMAHAN H B,AVENT B,et al.Advancesand open problems in federated learning[J].Foundations and Trends© in Machine Learning,2021,14(1/2):1-210.
[13]BARUCH G,BARUCH M,GOLDBERG Y.A little is enough:Circumventing defenses for distributed learning[C]//Procee-dings of the 33rd International Conference on Neural Information Processing Systems.2019:8635-8645.
[14]BAGDASARYAN E,VEIT A,HUA Y,et al,How to backdoor federated learning[C]// International Conference on Artificial Intelligence and Statistics.PMLR,2020:2938-2948.
[15]BLANCHARD P,EL MHAMDI E M,GUERRAOUI R,et al.Machine learning with adversaries:Byzantine tolerant gradient descent[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.2017:118-128.
[16]YIN D,CHEN Y,KANNAN R,et al.Byzantine-robust distributed learning:Towards optimal statistical rates[C]//International Conference on Machine Learning.PMLR,2018:5650-5659.
[17]CHEN Y,SU L,XU J.Distributed statistical machine learning in adversarial settings:Byzantine gradient descent[C]//Proceedings of the ACM on Measurement and Analysis of Computing Systems.2017:1-25.
[18]FUNG C,YOON C J M,BESCHASTNIKH I.The limitations of federated learning in sybil settings[C]//23^rd International Symposium on Research in Attacks,Intrusions and Defenses({RAID} 2020).2020:301-316.
[19]LI S,CHENG Y,WANG W,et al.Learning to detect malicious clients for robust federated learning[J].arXiv:2002.00211,2020.
[20]XIE C,KOYEJO S,GUPTA I.Zeno:Distributed stochastic gradient descent with suspicion-based fault-tolerance[C]//International Conference on Machine Learning.PMLR,2019:6893-6901.
[21]RODRÍGUEZ-BARROSO N,MARTÍNEZ-CÁMARA E,LUZ-ÓN M V,et al.Dynamic defense against byzantine poisoning attacks in federated learning[J].Future Generation Computer Systems,2022,133:1-9.
[22]GUERRAOUI R,ROUAULT S.The hidden vulnerability ofdistributed learning in Byzantium[C]//International Conference on Machine Learning.PMLR,2018:3521-3530.
[23]KHAZBAK Y,TAN T,CAO G.MLGuard:Mitigating poisoning attacks in privacy preserving distributed collaborative learning[C]//2020 29th International Conference on Computer Communications and Networks(ICCCN).IEEE,2020:1-9.
[24]LU Y,FAN L.An efficient and robust aggregation algorithm for learning federated cnn[C]//Proceedings of the 2020 3rd International Conference on Signal Processing and Machine Learning.2020:1-7.
[25]YU L,WU L.Towards byzantine-resilient federated learning via group-wise robust aggregation[J].Federated Learning:Privacy and Incentive,2020,12500:81-92.
[26]YANG H,ZHANG X,FANG M,et al.Byzantine-resilient stochastic gradient descent for distributed learning:A lipschitz-inspired coordinate-wise median approach[C]//IEEE 58th Conference on Decision and Control(CDC 2019).IEEE,2019:5832-5837.
[27]WANG Y,ZHU T,CHANG W,et al.Model poisoning defense on federated learning:A validation based approach[C]//International Conference on Network and System Security.Cham:Springer International Publishing,2020:207-223.
[28]TAN J,LIANG Y C,LUONG N C,et al.Toward smart security enhancement of federated learning networks[J].IEEE Network,2021,35(1):340-347.
[29]CHEN Z,TIAN P,LIAO W,et al.Zero knowledge clustering based adversarial mitigation in heterogeneous federated learning[J].IEEE Transactions on Network Science and Engineering,2020,8(2):1070-1083.
[30]KIM W,LIM H.FedCC:Federated Learning with ConsensusConfirmation for Byzantine Attack Resistance(Student Abstract)[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2022:12981-12982.
[31]CAO X,LAI L.Distributed gradient descent algorithm robust to an arbitrary number of byzantine attackers[J].IEEE Transactions on Signal Processing,2019,67(22):5850-5864.
[32]GU Z,HE L,LI P,et al.FREPD:A Robust Federated Learning Framework on Variational Autoencoder[J].Comput.Syst.Sci.Eng.,2021,39(3):307-320.
[33]ZHAI K,REN Q,WANG J,et al.Byzantine-robust federatedlearning via credibility assessment on Non-IID data[J].Mathematical Biosciences and Engineering,2022,19(2):1659-1676.
[34]SHAFAHI A,HUANG W R,NAJIBI M,et al.Poison frogs!Targeted clean-label poisoning attacks on neural networks[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems,2018:6106-6116.
[35]SHEJWALKAR V,HOUMANSADR A.Manipulating the byzantine:Optimizing model poisoning attacks and defenses for federated learning[C]//NDSS.2021.
[36]XIE C,KOYEJO O,GUPTA I.Fall of empires:Breaking byzantine-tolerant sgd by inner product manipulation[C]//Uncertainty in Artificial Intelligence.PMLR,2020:261-270.
[37]LIN J,DU M,LIU J.Free-riders in federated learning:Attacks and defenses[J].arXiv:1911.12560,2019.
[38]BHAGOJI A N,CHAKRABORTY S,MITTAL P,et al.Analyzing federated learning through an adversarial lens[C]//International Conference on Machine Learning.PMLR,2019:634-643.
[39]LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-basedlearning applied to document recognition[C]//Proceedings of the IEEE.1998:2278-2324.
[40]XIAO H,RASUL K,VOLLGRAF R.Fashion-mnist:a novelimage dataset for benchmarking machine learning algorithms[J].arXiv:1708.07747,2017.
[41]KRIZHEVSKY A,HINTON G.Learning multiple layers of features from tiny images[DB/OL].https://learning2hash.github.io/publications/cifar2009learning/.
[42]HSU T M H,QI H,BROWN M.Measuring the effects of non-identical data distribution for federated visual classification[J].arXiv:1909.06335,2019.
[43]DAVENPORT C.Gboard passes one billion installs on the play store[J/OL].https://www.androidpolice.com/2018/08/22/gboard-passes-one-billion-installs-play-store,accessed:2023-12-2.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed