计算机科学

计算机科学 ›› 2025, Vol. 52 ›› Issue (11): 434-443.doi: 10.11896/jsjkx.250100146

• 信息安全 • 上一篇    下一篇

基于知识蒸馏的联邦学习后门攻击方法

赵桐, 陈学斌, 王柳, 景忠瑞, 钟琪   

  1. 华北理工大学理学院 河北 唐山 063210
    河北省数据科学与应用重点实验室(华北理工大学) 河北 唐山 063210
    唐山市数据科学重点实验室(华北理工大学) 河北 唐山 063210
  • 收稿日期:2025-01-23 修回日期:2025-04-28 出版日期:2025-11-15 发布日期:2025-11-06
  • 通讯作者: 陈学斌(chxb@ncst.edu.cn)
  • 作者简介:(zhaot@stu.ncst.edu.cn)
  • 基金资助:
    国家自然科学基金(U20A20179)

Backdoor Attack Method for Federated Learning Based on Knowledge Distillation

ZHAO Tong, CHEN Xuebin, WANG Liu, JING Zhongrui, ZHONG Qi   

  1. College of Science,North China University of Science and Technology,Tangshan,Hebei 063210,China
    Hebei Province Key Laboratory of Data Science and Application(North China University of Science and Technology),Tangshan,Hebei 063210,China
    Tangshan Key Laboratory of Data Science(North China University of Science and Technology),Tangshan,Hebei 063210,China
  • Received:2025-01-23 Revised:2025-04-28 Online:2025-11-15 Published:2025-11-06
  • About author:ZHAO Tong,born in 1998,postgra-duate,is a member of CCF(No.W0214G).His main research interests include data security and federated learning.
    CHEN Xuebin,born in 1970,Ph.D,professor,is a outstanding member of CCF(No.13654D).His main research in-terests include big data security,Internet of security and network security.
  • Supported by:
    National Natural Science Foundation of China(U20A20179).

PDF (PC)

23

摘要： 联邦学习能够使不同参与者利用私人数据集共同训练一个全局模型。然而,联邦学习的分布式特性,也为后门攻击提供了空间。后门攻击中的攻击者对全局模型进行投毒,使全局模型在遇到带有特定后门触发器的样本时被误导至有针对性的错误预测。对此,提出了一种基于知识蒸馏的联邦学习后门攻击方法(KDFLBD)。首先,利用蒸馏生成的浓缩毒化数据集训练教师模型,并将教师模型的“暗知识”传递给学生模型,以提炼恶意神经元。然后,通过神经元Z分数排序和混合,将带有后门的神经元嵌入全局模型。在常见数据集上评估了KDFLBD在iid和non-iid场景下的性能,相较于像素攻击和标签翻转攻击,KDFLBD在保证主任务准确率(MTA)不受影响的同时,显著提升了攻击成功率(ASR)。

关键词: 联邦学习, 后门攻击, 知识蒸馏, 触发器, 隐私保护

Abstract: Federated learning enables different participants to jointly train a global model using their private datasets.However,the distributed nature of federated learning also provides room for backdoor attacks.The attacker of the backdoor attack poisons the global model causing the global model misleads to targeted incorrect predictions when encountering samples with specific backdoor triggers.This paper proposes a backdoor attack method for federated learning based on knowledge distillation.Firstly,the teacher model is trained using the concentrated poison dataset generated by distillation,and the “dark knowledge” of the teacher model is transferred to the student model to refine the maliciousneurons.Then,the neurons with backdoors are embedded into the global model through Z-scoreranking and mixing of neurons .The experiment is evaluated the performance of KDFLBD in iid and non-iid scenarios on common datasets.Compared with pixel attacks and label flipping attacks,KDFLBD significantly improves the attack success rate(ASR) while ensuring that the main task accuracy(MTA) is not affected.

Key words: Federated learning, Backdoor attack, Knowledge distillation, Trigger, Privacy protection

中图分类号: 

  • TP391
[1]MOORE I N,SNYDER S L,MILLER C,et al.Confidentialityand Privacy in Health Care from the Patient's Perspective:Does HIPPA Help?[J].Health Matrix,2007,17:215.
[2]VOIGT P,VON DEM BUSSCHE A.The eu general data protection regulation(gdpr):A Practical Guide(1st Ed.)[M].Cham:Springer International Publishing,2017.
[3]CHENG X.On the personal information processing rules in our country's personal information protection law [J].Tsinghua Law,2021,15(3):55-73.
[4]MCMAHAN B,MOORE E,RAMAGE D,et al.Communica-tion-efficient learning of deep networks from decentralized data[C]//Artificial Intelligence and Statistics.PMLR,2017:1273-1282.
[5]BAGDASARYAN E,VEIT A,HUA Y,et al.How to backdoor federated learning[C]//International Conference on Artificial Intelligence and Statistics.PMLR,2020:2938-2948.
[6]XUE M,NI S,WU Y,et al.Imperceptible and multi-channelbackdoor attack[J].Applied Intelligence,2024,54(1):1099-1116.
[7]BAGDASARYAN E,SHMATIKOV V.Blind backdoors in deep learning models[C]//30th USENIX Security Symposium(USENIX Security 21).2021:1505-1521.
[8]RAWAT A,LEVACHER K,SINN M.The devil is in theGAN:backdoor attacks and defenses in deep generative models[C]//European Symposium on Research in Computer Security.Cham:Springer Nature Switzerland,2022:776-783.
[9]NGUYEN T D,RIEGER P,MIETTINEN M,et al.Poisoningattacks on federated learning-based IoT intrusion detection system[C]//Proc.Workshop Decentralized IoT Syst.Secur.(DISS).2020:1-7.
[10]LIU Y,GARG S,NIE J,et al.Deep anomaly detection for time-series data in industrial iot:A communication-efficient on-device federated learning approach[J].IEEE Internet of Things Journal,2021(8):6348-6358.
[11]CHEN M,SURESH A T,MATHEWS R,et al.Federatedlearning of n-gram language models[J].arXiv:1910.03432,2019.
[12]LI T,SAHU A K,ZAHEER M,et al.Federated optimization in heterogeneous networks[C]//Proceedings of Machine Learning and Systems.2020:429-450.
[13]LI X,JIANG M,ZHANG X,et al.Fedbn:Federated learning on non-iid features via local batch normalization[J].arXiv:2102.07623,2021.
[14]LI Q,HE B,SONG D.Model-contrastive federated learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:10713-10722.
[15]GU T,DOLAN-GAVITT B,GARG S.Badnets:Identifying vulnerabilities in the machine learning model supply chain[J].ar-Xiv:1708.06733,2017.
[16]ALBERTI M,PONDENKANDATH V,WURSCH M,et al.Are you tampering with my data?[C]//Proceedings of the Euro-pean Conference on Computer Vision(ECCV).2018.
[17]BARNI M,KALLAS K,TONDI B.A new backdoor attack in cnns by training set corruption without label poisoning[C]//2019 IEEE International Conference on Image Processing(ICIP).IEEE,2019:101-105.
[18]XIAO Q,CHEN Y,SHEN C,et al.Seeing is not believing:Camouflage attacks on image scaling algorithms[C]//28th USENIX Security Symposium(USENIX Security 19).2019:443-460.
[19]LI Y,LI Y,WU B,et al.Invisible backdoor attack with sample-specific triggers[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:16463-16472.
[20]SHAFAHI A,HUANG W R,NAJIBI M,et al.Poison frogs! targeted clean-label poisoning attacks on neural networks[J].arXiv:1804.00792,2018.
[21]GAO Y,LI Y,ZHU L,et al.Not all samples are born equal:Towards effective clean-label backdoor attacks[J].Pattern Recognition,2023,139:109512.
[22]LIN J,XU L,LIU Y,et al.Composite backdoor attack for deep neural network by mixing existing benign features[C]//Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security.2020:113-131.
[23]WANG H,SREENIVASAN K,RAJPUT S,et al.Attack of the tails:Yes,you really can backdoor federated learning[J].Advances in Neural Information Processing Systems,2020,33:16070-16084.
[24]YOO K Y,KWAK N.Backdoor attacks in federated learning by rare embeddings and gradient ensembling[J].arXiv:2204.14017,2022.
[25]ZHANG J,CHEN B,CHENG X,et al.PoisonGAN:Generative poisoning attacks against federated learning in edge computing systems[J].IEEE Internet of Things Journal,2020,8(5):3310-3322.
[26]GONG X,CHEN Y,HUANG H,et al.Coordinated backdoor attacks against federated learning with model-dependent triggers[J].IEEE Network,2022,36(1):84-90.
[27]XIE C,HUANG K,CHEN P Y,et al.Dba:Distributed backdoor attacks against federated learning[C]//International Conference on Learning Representations.2019.
[28]SUN Z,KAIROUZ P,SURESH A T,et al.Can you really backdoor federated learning?[J].arXiv:1911.07963,2019.
[29]LIU Y,YI Z,CHEN T.Backdoor attacks and defenses in feature-partitioned collaborative learning[J].arXiv:2007.03608,2020.
[30]ZHOU X,XU M,WU Y,et al.Deep model poisoning attack on federated learning[J].Future Internet,2021,13(3):73.
[31]ZHANG Z,PANDA A,SONG L,et al.Neurotoxin:Durable backdoors in federated learning[C]//International Conference on Machine Learning.PMLR,2022:26429-26446.
[32]BUCILUĂ C,CARUANA R,NICULESCU-MIZIL A.Model compression[C]//Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mi-ning.2006:535-541.
[33]HINTON G,VINYALS O,DEAN J.Distilling the knowledge in a neural network[J].arXiv:1503.02531,2015.
[34]CAZENAVETTE G,WANG T,TORRALBA A,et al.Dataset distillation by matching training trajectories[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:4750-4759.
[35]NGUYEN T,CHEN Z,LEE J.Dataset meta-learning from kernel ridge-regression[J].arXiv:2011.00050,2020.
[36]NGUYEN T,NOVAK R,XIAO L,et al.Dataset distillationwith infinitely wide convolutional networks[J].Advances in Neural Information Processing Systems,2021,34:5186-5198.
[37]ZHAO B,BILEN H.Dataset condensation with differentiable siamese augmentation[C]//International Conference on Machine Learning.PMLR,2021:12674-12685.
[38]ZHAO B,MOPURI K R,BILEN H.Datasetcondensation with gradient matching[J].arXiv:2006.05929,2020.
[39]WANG T,ZHU J Y,TORRALBA A,et al.Dataset distillation[J].arXiv:1811.10959,2018.
[40]RUBINSTEIN R.The cross-entropy method for combinatorial and continuous optimization[J].Methodology and Computing in Applied Probability,1999,1(2):127-190.
[41]LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-basedlearning applied to document recognition[C]//Proceedings of the IEEE.2002:2278-2324.
[42]XIAO H,RASUL K,VOLLGRAF R.Fashion-mnist:a novel image dataset for benchmarking machine learning algorithms[J].arXiv:1708.07747,2017.
[43]KRIZHEVSKY A,HINTON G.Learning multiple layers of features from tiny images[J/OL].https://www.cs.utoronto.ca/~kriz/learning-features-2009-TR.pdf.
[44]CAO X,JIA J,GONG N Z.Provably secure federated learning against malicious clients[C]//Proceedings of the AAAI Confe-rence on Artificial Intelligence.2021:6885-6893.
[45]KRIZHEVSKY A,SUTSKEVER I,HINTON G E.Imagenetclassification with deep convolutional neural networks[J/OL].https://proceedings.neurips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf.
[46]NGUYEN T D,NGUYEN T,LE NGUYEN P,et al.Backdoor attacks and defenses in federated learning:Survey,challenges and future research directions[J].Engineering Applications of Artificial Intelligence,2024,127:107166.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发