Computer Science ›› 2024, Vol. 51 ›› Issue (11A): 240100176-9.doi: 10.11896/jsjkx.240100176

• Information Security • Previous Articles     Next Articles

Multi-party Co-governance Prevention Strategy for Horizontal Federated Learning Backdoors

XU Wentao1, WANG Binjun1, ZHU Lixin2, WANG Hanxu1, GONG Ying1   

  1. 1 College of Information and Cyber Security,People's Public Security University of China,Beijing 100038,China
    2 Suzhou Institute of Information Security Law,Xi'an Jiaotong University,Suzhou,Jiangsu 215123,China
  • Online:2024-11-16 Published:2024-11-13
  • About author:XU Wentao,born in 1999,postgraduate.His main research interests include federated learning and backdoor attack.
    WANG Binjun,born in 1962,Ph.D,professor,Ph.D supervisor,is a member of CCF(No.E200014787M).His main research interests include network security and law enforcement.
  • Supported by:
    Key Program of National Social Science Foundation(20AZD114).

Abstract: Federated learning is susceptible to backdoor attacks based on model replacement.In response to the poor performance of current backdoor detection methods,multi-party co-governance prevention strategy is proposed.The aim is to establish a co-go-vernance mechanism between the federated learning center server and the client,so as to effectively detect and prevent backdoors in the model without compromising data privacy and main task performance.This strategy covers shallow backdoor scanning,deep backdoor detection,and model repair,all of which are completed by the client in collaboration with the central server.Among them,shallow backdoor scanning is a lightweight real-time backdoor detection scheme that does not significantly increase time overhead.This scheme captures abnormal changes in the aggregated model parameters by the client and reports them to the central server.When the number of reports reaches the set threshold,the central server initiates deep backdoor detection,and each client pauses the federated learning process for deep detection to determine whether the neurons in the model are affected by backdoor attacks and exhibit abnormalities.If there are anomalies,each client adopts a method of concatenating a benign model and an attacked model to restore the model to a benign state,and submits the results of deep backdoor detection and model repair plans to the central server.It is up to the central server to decide the final repair plan,thereby thoroughly clearing the backdoor.Experimental results show that this strategy can effectively detect and remove backdoors in the federated learning model,ensuring the safe operation of horizontal federated learning.

Key words: Federated learning, Backdoor attack, Backdoor detection, Multi-party co-governance

CLC Number: 

  • TP391
[1]CHEN H L,FU C,ZHAO J S,et al.DeepInspect:A Black-box Trojan Detection and Mitigation Framework for Deep Neural Networks[C]//IJCAI.2019:4658-4664.
[2]MCMAHAN B,MOORE E,RAMAGE D,et al.Communica-tion-efficient learning of deep networks from decentralized data[C]//Artificial Intelligence and Statistics.PMLR,2017:1273-1282.
[3]KAIROUZ P,MCMAHAN H B,AVENT B,et al.Advancesand open problems in federated learning[J].Foundations and Trends in Machine Learning,2021,14(1/2):1-210.
[4]GU T Y,DOLAN-GAVITT B,GARG S.Badnets:Identifyingvulnerabilities in the machine learning model supply chain[J].arXiv:1708.06733,2017.
[5]BAGDASARYAN E,VEIT A,HUA Y,et al.How to backdoorfederated learning[C]//International Conference on Artificial Intelligence and Statistics.New York:PMLR,2020:2938-2948.
[6]WANG H,SREENIVASAN K,RAJPUT S,et al.Attack of thetails:Yes,you really can backdoor federated learning[J].Advances in Neural Information Processing Systems,2020,33:16070-16084.
[7]GU T,LIU K,DOLAN-GAVITT B,et al.Badnets:Evaluatingbackdooring attacks on deep neural networks[J].IEEE Access,2019,7:47230-47244.
[8]CHEN X,LIU C,LI B,et al.Targeted backdoor attacks on deep learning systems using data poisoning[J].arXiv:1712.05526,2017.
[9]BONAWITZ K,IVANOV V,KREUTER B,et al.Practical se-cure aggregation for privacy-preserving machine learning[C]//Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security.New York:ACM,2017:1175-1191.
[10]SUN Z,KAIROUZ P,SURESH A T,et al.Can you really backdoor federated learning?[J].arXiv:1911.07963,2019.
[11]XU W T,WANG B J.Backdoor Defense of Horizontal Federated Learning Based on Random Cutting and Gradient Clipping[J].Computer Science,2023,50(11):356-363.
[12]YIN D,CHEN Y,KANNAN R,et al.Byzantine-robust distri-buted learning:Towards optimal statistical rates[C]//International Conference on Machine Learning.PMLR,2018:5650-5659.
[13]MI Y,GUAN J,ZHOU S.Ariba:Towards accurate and robust identification of backdoor attacks in federated learning[J].ar-Xiv:2202.04311,2022.
[14]FUNG C,YOON C J M,BESCHASTNIKH I.The limitations offederated learning in sybil settings[C]//23rd International Symposium on Research in Attacks,Intrusions and Defenses(RAID 2020).2020:301-316.
[15]ANDREINA S,MARSON G A,MÖLLERING H,et al.Baffle:Backdoor detection via feedback-based federated learning[C]//2021 IEEE 41st International Conference on Distributed Computing Systems(ICDCS).IEEE,2021:852-863.
[16]WANG B,YAO Y,SHAN S,et al.Neural cleanse:Identifyingand mitigating backdoor attacks in neural networks[C]//2019 IEEE Symposium on Security and Privacy(SP).IEEE,2019:707-723.
[17]GEIPING J,BAUERMEISTER H,DRÖGE H,et al.Inverting gradients-how easy is it to break privacy in federated learning?[J].Advances in Neural Information Processing Systems,2020,33:16937-16947.
[18]ZHU L,HAN S.Deep leakage from gradients[C]//Advances in Neural Information Processing Systems 32:Annual Conference on Neural Information Processing Systems.New York:Curran Associates Inc,2019:14747-14756.
[19]MCMAHAN B,MOORE E,RAMAGE D,et al.Communica-tion-efficient learning of deep networks from decentralized data[C]//Artificial Intelligence and Statistics.Florida:PMLR,2017:1273-1282.
[20]FANG H,QIAN Q.Privacy preserving machine learning with homomorphic encryption and federated learning[J].Future Internet,2021,13(4):94.
[21]LEYS C,LEY C,KLEIN O,et al.Detecting outliers:Do not use standard deviation around the mean,use absolute deviation around the median[J].Journal of Experimental Social psycho-logy,2013,49(4):764-766.
[22]ZOPH B,LE Q V.Neural architecture search with reinforce-ment learning[J].arXiv:1611.01578,2016.
[23]ESTER M,KRIEGEL H P,SANDER J,et al.A density-basedalgo-rithm for discovering clusters in large spatial databases with noise[C]//KDD.1996:226-231.
[24]GARBER L.Denial-of-service attacks rip the Internet[J].Computer,2000,33(4):12-17.
[25]FANG M,CAO X,JIA J,et al.Local model poisoning attacks to {Byzantine-Robust} federated learning[C]//29th USENIX security symposium(USENIX Security 20).2020:1605-1622.
[26]XIE C,KOYEJO O,GUPTA I.Fall of empires:Breaking byz-antine-tolerant sgd by inner product manipulation[C]//Uncertainty in Artificial Intelligence.PMLR,2020:261-270.
[27]BLANCHARD P,EL MHAMDI E M,GUERRAOUI R,et al.Machine learning with adversaries:Byzantine tolerant gradient descent[J].Advances in Neural Information Processing Systems,2017,30.
[28]FUNG C,YOON C J M,BESCHASTNIKH I.The limitations of federated learning in sybil settings[C]//23rd International Symposium on Research in Attacks[C]//Intrusions and Defenses(RAID 2020).2020:301-316.
[29]GU T,DOLAN-GAVITT B,GARG S.Badnets:Identifying vul-nerabilities in the machine learning model supply chain[J].ar-Xiv:1708.06733,2017.
[30]KRIZHEVSKY A,HINTON G.Learning multiple layers of fea-tures from tiny images[J].Handbook of Systemic Autoim-mune Diseases,2009,1(4).
[1] LI Zhi, LIN Sen, ZHANG Qiang. Edge Cloud Computing Approach for Intelligent Fault Detection in Rail Transit [J]. Computer Science, 2024, 51(9): 331-337.
[2] GAN Run, WEI Xianglin, WANG Chao, WANG Bin, WANG Min, FAN Jianhua. Backdoor Attack Method in Autoencoder End-to-End Communication System [J]. Computer Science, 2024, 51(7): 413-421.
[3] ZHOU Tianyang, YANG Lei. Study on Client Selection Strategy and Dataset Partition in Federated Learning Basedon Edge TB [J]. Computer Science, 2024, 51(6A): 230800046-6.
[4] SUN Min, DING Xining, CHENG Qian. Federated Learning Scheme Based on Differential Privacy [J]. Computer Science, 2024, 51(6A): 230600211-6.
[5] TAN Zhiwen, XU Ruzhi, WANG Naiyu, LUO Dan. Differential Privacy Federated Learning Method Based on Knowledge Distillation [J]. Computer Science, 2024, 51(6A): 230600002-8.
[6] LIU Dongqi, ZHANG Qiong, LIANG Haolan, ZHANG Zidong, ZENG Xiangjun. Study on Smart Grid AMI Intrusion Detection Method Based on Federated Learning [J]. Computer Science, 2024, 51(6A): 230700077-8.
[7] WANG Chenzhuo, LU Yanrong, SHEN Jian. Study on Fingerprint Recognition Algorithm for Fairness in Federated Learning [J]. Computer Science, 2024, 51(6A): 230800043-9.
[8] ZANG Hongrui, YANG Tingting, LIU Hongbo, MA Kai. Study on Cryptographic Verification of Distributed Federated Learning for Internet of Things [J]. Computer Science, 2024, 51(6A): 230700217-5.
[9] LIU Jianxun, ZHANG Xinglin. Federated Learning Client Selection Scheme Based on Time-varying Computing Resources [J]. Computer Science, 2024, 51(6): 354-363.
[10] XU Yicheng, DAI Chaofan, MA Wubin, WU Yahui, ZHOU Haohao, LU Chenyang. Particle Swarm Optimization-based Federated Learning Method for Heterogeneous Data [J]. Computer Science, 2024, 51(6): 391-398.
[11] LU Yanfeng, WU Tao, LIU Chunsheng, YAN Kang, QU Yuben. Survey of UAV-assisted Energy-Efficient Edge Federated Learning [J]. Computer Science, 2024, 51(4): 270-279.
[12] WANG Degang, SUN Yi, GAO Qi. Active Membership Inference Attack Method Based on Multiple Redundant Neurons [J]. Computer Science, 2024, 51(4): 373-380.
[13] WANG Xin, HUANG Weikou, SUN Lingyun. Survey of Incentive Mechanism for Cross-silo Federated Learning [J]. Computer Science, 2024, 51(3): 20-29.
[14] HUANG Nan, LI Dongdong, YAO Jia, WANG Zhe. Decentralized Federated Continual Learning Method Combined with Meta-learning [J]. Computer Science, 2024, 51(3): 271-279.
[15] CHEN Jinyin, LI Xiao, JIN Haibo, CHEN Ruoxi, ZHENG Haibin, LI Hu. CheatKD:Knowledge Distillation Backdoor Attack Method Based on Poisoned Neuronal Assimilation [J]. Computer Science, 2024, 51(3): 351-359.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!