Computer Science ›› 2025, Vol. 52 ›› Issue (6A): 240500132-7.doi: 10.11896/jsjkx.240500132

• Information Security • Previous Articles     Next Articles

Federated Learning Privacy Protection Method Combining Dataset Distillation

WANG Chundong, ZHANG Qinghua, FU Haoran   

  1. School of Computer Science and Engineering,Tianjin University of Technology,Tianjin 300384,China
    National Engineering Laboratory of Computer Virus Prevention and Control Technology,Tianjin 300384,China
  • Online:2025-06-16 Published:2025-06-12
  • About author:WANG Chundong,born in 1969,Ph.D,professor,is a member of CCF(No.16230M).His main research interests include big data and smart computing security,network security situation awareness,etc.
  • Supported by:
    National Natural Science Foundation of China(U1536122),Joint Funds of the Tianjin Municipal Commission of Education,China(2021YJSB252) and Science and Technology Commission Major Special Projects of Tianjin,China(15ZXDSGX00030).

Abstract: Federated learning trains a global model by exchanging model parameters rather than data,with the goal of achieving privacy protection.However,a large number of studies have shown that attackers can infer the original training data through intercepted gradients,leading to privacy leakage on clients.In addition,the different sampling methods of different clients can lead to the phenomenon of non independent and identically distributed collected data,which can affect the overall training performance of the model.To cope with gradient inversion attacks,the data distillation method is introduced into the federated learning framework,while combining data augmentation methods to enhance the availability of synthesized data.In addition,to address the issue of data heterogeneity in medical data from different institutions,a batch normalization layer is introduced into the client to alleviate client drift and improve the overall performance of the model.Experimental results indicate that while achieving similar performance to other federated learning paradigms,the federated learning method combined with data distillation also enhances the protection of medical data privacy.

Key words: Federated learning, Privacy protection, Data distillation, Image classification, Data heterogeneity

CLC Number: 

  • TP183
[1]MCMAHAN B,MOORE E,RAMAGE D,et al.Communica-tion-efficient learning of deep networks from decentralized data[C]//Artificial Intelligence and Statistics.PMLR,2017:1273-1282.
[2]GUO P,WANG P,ZHOU J,et al.Multi-institutional collaborations for improving deep learning-based magnetic resonance image reconstruction using federated learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:2423-2432.
[3]KUMAR R,KHAN AA,KUMAR J,et al.Blockchain-federated-learning and deep learning models for covid-19 detection using ct imaging[J].IEEE Sensors Journal,2021,21(14):16301-16314.
[4]ZHU L,LIU Z,HAN S.Deep leakage from gradients[J].Advances in Neural Information Processing Systems,2019,32.
[5]ZHAO B,MOPURI K R,BILEN H.idlg:Improved deep leakage from gradients[J].arXiv:2001.02610,2020.
[6]GEIPIN J,BAUERMEISTER H,DRÖGE H,et al.Invertinggradients-how easy is it to break privacy in federated learning?[J].Advances in Neural Information Processing Systems,2020,33:16937-16947.
[7]WAINAKH A,VENTOLA F,MÜßIG T,et al.User-level label leakage from gradients in federated learning[J].arXiv:2105.09369,2021.
[8]YIN H,MALLYA A,VAHDAT A,et al.See through gra-dients:Image batch recovery viagradinversion[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:16337-16346.
[9]WEI W,LIU L,WU Y,et al.Gradient-leakage resilient federated learning[C]//2021 IEEE 41st International Conference on Distributed Computing Systems(ICDCS).IEEE,2021:797-807.
[10]KU H,SUSILO W,ZHANG Y,et al.Privacy-preserving federated learning in medical diagnosis with homomorphic re-encryption[J].Computer Standards & Interfaces,2022,80:103583.
[11]IOFFE S,SZEGEDY C.Batch normalization:Accelerating deep network training by reducing internal covariate shift[C]//International Conference on Machine Learning.pmlr,2015:448-456.
[12]SUN G,CONG Y,DONG J,et al.Data poisoning attacks on fe-derated machine learning[J].IEEE Internet of Things Journal,2021,9(13):11365-11375.
[13]FANG M,CAO X,JIA J,et al.Local model poisoning attacks to {Byzantine-Robust} federated learning[C]//29th USENIX Security Symposium(USENIX Security 20).2020:1605-1622.
[14]BERNSTEIN J,ZHAO J,AZIZZADENESHELI K,et al.signSGD with Majority Vote is Communication Efficient And Byzantine Fault Tolerant.CoRR abs/1810.05291(2018)[J].arXiv:1810.05291,2018.
[15]WANG H,SREENIVASAN K,RAJPUT S,et al.Attack of the tails:Yes,you really can backdoor federatedlearning[J].Advances in Neural Information Processing Systems,2020,33:16070-16084.
[16]XIE C,HUANG K,CHEN P Y,et al.Dba:Distributed backdoor attacks against federated learning[C]//International Conference on Learning Representations.2019.
[17]BAGDASARYAN E,VEIT A,HUA Y,et al.How to backdoor federated learning[C]//International Conference on Artificial Intelligence and Statistics.PMLR,2020:2938-2948.
[18]LYU L,CHEN C.A Novel Attribute Reconstruction Attack in Federated Learning[J].arXiv:2108.06910,2021.
[19]MELIS L,SONG C,DE CRISTOFARO E,et al.Exploiting unintended feature leakage in collaborative learning[C]//2019 IEEE Symposium on Security and Privacy(SP).IEEE,2019:691-706.
[20]LU Y,HUANG X,DAI Y,et al.Blockchain and federated learning for privacy-preserved data sharing in industrial IoT[J].IEEE Transactions on Industrial Informatics,2019,16(6):4177-4186.
[21]PARK J,LIM H.Privacy-preserving federated learning usinghomomorphic encryption[J].Applied Sciences,2022,12(2):734.
[22]LI T,SAHU A K,ZAHEER M,et al.Federated optimization in heterogeneous networks[C]//Proceedings of Machine Learning and Systems.2020:429-450.
[23]ARIVAZHAGAN M G,AGGARWAL V,SINGH A K,et al.Federated learning with personalization layers[J].arXiv:1912.00818,2019.
[24]TDINH C,TRAN N,NGUYEN J.Personalized federatedlearning withmoreau envelopes[J].Advances in Neural Information Processing Systems,2020,33:21394-21405.
[25]ZHAO B,MOPURI K R,BILEN H.Dataset condensation with gradient matching[J].arXiv:2006.05929,2020.
[26]ZHAO B,BILEN H.Dataset condensation with differentiablesiamese augmentation[C]//International Conference on Machine Learning.PMLR,2021:12674-12685.
[27]LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-basedlearning applied to document recognition[C]//Proceedings of the IEEE.1998:2278-2324.
[28]KRIZHEVSKY A,HINTON G.Learning multiple layers of features from tiny images[J].Handbook of Systemic Autoimmune Diseases,2009,1(4).
[29]CHOWDHURY M E H,RAHMAN T,KHANDAKAR A,et al.Can AI help in screening viral and COVID-19 pneumonia?[J].IEEE Access,2020,8:132665-132676.
[30]RAHMAN T,KHANDAKAR A,QIBLAWEY Y,et al.Exploring the effect of image enhancement techniques on COVID-19 detection using chest X-ray images[J].Computers in Biology and Medicine,2021,132:104319.
[31]YANG J,SHI R,NI B.Medmnist classification decathlon:Alightweight automl benchmark for medical image analysis[C]//2021 IEEE 18th International Symposium on Biomedical Imaging(ISBI).IEEE,2021:191-195.
[32]WANG T,ZHU J Y,TORRALBA A,et al.Dataset distillation[J].arXiv:1811.10959,2018.
[33]BOHDAL O,YANG Y,HOSPEDALES T.Flexible dataset distillation:Learn labels instead of images[J].arXiv:2006.08572,2020.
[34]LI X,JIANG M,ZHANG X,et al.Fedbn:Federated learning on non-iid features via local batch normalization[J].arXiv:2102.07623,2021.
[35]HSU T M H,QI H,BROWN M.Measuring the effects of non-identical data distribution for federated visual classification[J].arXiv:1909.06335,2019.
[36]GUO S,YANG X,FENG J,et al.FedGR:Federated Learningwith Gravitation Regulation for Double Imbalance Distribution[C]//International Conference on Database Systems for Advanced Applications.Cham:Springer Nature Switzerland,2023:703-718.
[37]YUROCHKIN M,AGARWAL M,GHOSH S,et al.Bayesian nonparametric federated learning of neural networks[C]//International Conference on Machine Learning.2019:7252- 7261.
[1] LIU Runjun, XIAO Fengjun, HU Weitong, WANG Xu. Reversible Data Hiding in Fully Encrypted Images Based on Pixel Interval Partitioning andPrediction Recovery [J]. Computer Science, 2025, 52(6A): 240900030-8.
[2] LEI Shuai, QIU Mingxin, LIU Xianhui, ZHANG Yingyao. Image Classification Model for Waste Household Appliance Recycling Based on Multi-scaleDepthwise Separable ResNet [J]. Computer Science, 2025, 52(6A): 240500057-7.
[3] CHEN Yadang, GAO Yuxuan, LU Chuhan, CHE Xun. Saliency Mask Mixup for Few-shot Image Classification [J]. Computer Science, 2025, 52(6): 256-263.
[4] YUAN Lin, HUANG Ling, HAO Kaile, ZHANG Jiawei, ZHU Mingrui, WANG Nannan, GAO Xinbo. Adversarial Face Privacy Protection Based on Makeup Style Patch Activation [J]. Computer Science, 2025, 52(6): 405-413.
[5] SUN Jinyong, WANG Xuechun, CAI Guoyong, SHANG Zhiliang. Open Set Recognition Based on Meta Class Incremental Learning [J]. Computer Science, 2025, 52(5): 187-198.
[6] CAO Tengfei, YIN Runtian, ZHU Liang, XU Changqiao. Survey of Personalized Location Privacy Protection Technologies [J]. Computer Science, 2025, 52(5): 307-321.
[7] ZHENG Xu, HUANG Xiangjie, YANG Yang. Reversible Facial Privacy Protection Method Based on “Invisible Masks” [J]. Computer Science, 2025, 52(5): 384-391.
[8] SUN Tanghui, ZHAO Gang, GUO Meiqian. Long-tail Distributed Medical Image Classification Based on Large Selective Nuclear Bilateral-branch Networks [J]. Computer Science, 2025, 52(4): 231-239.
[9] WANG Yifei, ZHANG Shengjie, XUE Dizhan, QIAN Shengsheng. Self-supervised Backdoor Attack Defence Method Based on Poisoned Classifier [J]. Computer Science, 2025, 52(4): 336-342.
[10] JIANG Yufei, TIAN Yulong, ZHAO Yanchao. Persistent Backdoor Attack for Federated Learning Based on Trigger Differential Optimization [J]. Computer Science, 2025, 52(4): 343-351.
[11] LUO Zhengquan, WANG Yunlong, WANG Zilei, SUN Zhenan, ZHANG Kunbo. Study on Active Privacy Protection Method in Metaverse Gaze Communication Based on SplitFederated Learning [J]. Computer Science, 2025, 52(3): 95-103.
[12] HU Kangqi, MA Wubin, DAI Chaofan, WU Yahui, ZHOU Haohao. Federated Learning Evolutionary Multi-objective Optimization Algorithm Based on Improved NSGA-III [J]. Computer Science, 2025, 52(3): 152-160.
[13] WANG Ruicong, BIAN Naizheng, WU Yingjun. FedRCD:A Clustering Federated Learning Algorithm Based on Distribution Extraction andCommunity Detection [J]. Computer Science, 2025, 52(3): 188-196.
[14] WANG Dongzhi, LIU Yan, GUO Bin, YU Zhiwen. Edge-side Federated Continuous Learning Method Based on Brain-like Spiking Neural Networks [J]. Computer Science, 2025, 52(3): 326-337.
[15] XIE Jiachen, LIU Bo, LIN Weiwei , ZHENG Jianwen. Survey of Federated Incremental Learning [J]. Computer Science, 2025, 52(3): 377-384.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!